Addressing Token Uniformity in Transformers via Singular Value
Transformation

Addressing Token Uniformity in Transformers via Singular Value Transformation

24 August 2022

Yulan He

Papers citing "Addressing Token Uniformity in Transformers via Singular Value Transformation"

11 / 11 papers shown

Title
BioNCERE: Non-Contrastive Enhancement For Relation Extraction In Biomedical Texts Farshad Noravesh 34 0 0 31 Oct 2024
Generalized Probabilistic Attention Mechanism in Transformers DongNyeong Heo Heeyoul Choi 54 0 0 21 Oct 2024
Encourage or Inhibit Monosemanticity? Revisit Monosemanticity from a Feature Decorrelation Perspective Hanqi Yan Yanzheng Xiang Guangyi Chen Yifei Wang Lin Gui Yulan He 37 5 0 25 Jun 2024
Setting the Record Straight on Transformer Oversmoothing G. Dovonon M. Bronstein Matt J. Kusner 20 5 0 09 Jan 2024
Why "classic" Transformers are shallow and how to make them go deep Yueyao Yu Yin Zhang ViT 16 0 0 11 Dec 2023
Graph Convolutions Enrich the Self-Attention in Transformers! Jeongwhan Choi Hyowon Wi Jayoung Kim Yehjin Shin Kookjin Lee Nathaniel Trask Noseong Park 25 4 0 07 Dec 2023
Extract-and-Adaptation Network for 3D Interacting Hand Mesh Recovery J. Park Daniel Sungho Jung Gyeongsik Moon Kyoung Mu Lee 16 6 0 05 Sep 2023
Addressing the Rank Degeneration in Sequential Recommendation via Singular Spectrum Smoothing Ziwei Fan Zhiwei Liu Hao Peng Philip S. Yu 30 1 0 21 Jun 2023
Unsupervised Speech Representation Pooling Using Vector Quantization J. Park Kwanghee Choi Hyunjun Heo Hyung-Min Park SSL 17 0 0 08 Apr 2023
Distinguishability Calibration to In-Context Learning Hongjing Li Hanqi Yan Yanran Li Li Qian Yulan He Lin Gui 19 2 0 13 Feb 2023
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,950 0 20 Apr 2018