Learning to Remove: Towards Isotropic Pre-trained BERT Embedding

v1v2 (latest)

Learning to Remove: Towards Isotropic Pre-trained BERT Embedding

International Conference on Artificial Neural Networks (ICANN), 2021

12 April 2021

ArXiv (abs)PDF HTML

Papers citing "Learning to Remove: Towards Isotropic Pre-trained BERT Embedding"

18 / 18 papers shown

Title
Vec2Summ: Text Summarization via Probabilistic Sentence Embeddings Mao Li Fred Conrad Johann Gagnon-Bartsch 61 0 0 09 Aug 2025
ALF: Advertiser Large Foundation Model for Multi-Modal Advertiser Understanding Santosh Rajagopalan Jonathan Vronsky Songbai Yan S. Alireza Golestaneh Shubhra Chandra Min Zhou 294 0 0 26 Apr 2025
Norm of Mean Contextualized Embeddings Determines their VarianceInternational Conference on Computational Linguistics (COLING), 2024 Hiroaki Yamagiwa Hidetoshi Shimodaira 134 0 0 17 Sep 2024
Exploring the Impact of a Transformer's Latent Space Geometry on Downstream Task Performance Anna C. Marbut John W. Chandler Travis J. Wheeler 222 1 0 18 Jun 2024
Revisiting Cosine Similarity via Normalized ICA-transformed Embeddings Hiroaki Yamagiwa Momose Oyama Hidetoshi Shimodaira LLMSV 213 5 0 16 Jun 2024
Rethinking ASTE: A Minimalist Tagging Scheme Alongside Contrastive Learning Qiao Sun Liujia Yang Minghao Ma Nanyang Ye Qinying Gu 374 2 0 12 Mar 2024
Isotropy, Clusters, and ClassifiersAnnual Meeting of the Association for Computational Linguistics (ACL), 2024 Timothee Mickus Stig-Arne Gronroos Joseph Attieh 238 0 0 05 Feb 2024
Outlier Dimensions Encode Task-Specific KnowledgeConference on Empirical Methods in Natural Language Processing (EMNLP), 2023 William Rudman Catherine Chen Carsten Eickhoff 215 8 0 26 Oct 2023
Impact of time and note duration tokenizations on deep learning symbolic music modelingInternational Society for Music Information Retrieval Conference (ISMIR), 2023 Nathan Fradet Nicolas Gutowski F. Chhel Jean-Pierre Briot 158 9 0 12 Oct 2023
Using Sequences of Life-events to Predict Human LivesNature Computational Science (Nat. Comput. Sci.), 2023 Germans Savcisens Tina Eliassi-Rad L. K. Hansen L. Mortensen Lau Lilleholt Anna Rogers Ingo Zettler Sune Lehmann AI4TS 201 66 0 05 Jun 2023
Exploring Anisotropy and Outliers in Multilingual Language Models for Cross-Lingual Semantic Sentence SimilarityAnnual Meeting of the Association for Computational Linguistics (ACL), 2023 Katharina Hämmerl Alina Fastowski Jindrich Libovický Kangyang Luo 329 10 0 01 Jun 2023
Stable Anisotropic RegularizationInternational Conference on Learning Representations (ICLR), 2023 William Rudman Carsten Eickhoff 241 10 0 30 May 2023
Improving Position Encoding of Transformers for Multivariate Time Series ClassificationData mining and knowledge discovery (DMKD), 2023 Navid Mohammadi Foumani Chang Wei Tan Geoffrey I. Webb Mahsa Salehi AI4TS 176 134 0 26 May 2023
Byte Pair Encoding for Symbolic MusicConference on Empirical Methods in Natural Language Processing (EMNLP), 2023 Nathan Fradet Nicolas Gutowski F. Chhel Jean-Pierre Briot 166 23 0 27 Jan 2023
Reliable Measures of Spread in High Dimensional Latent SpacesInternational Conference on Machine Learning (ICML), 2022 Anna C. Marbut Katy McKinney-Bock Travis J. Wheeler 267 3 0 15 Dec 2022
Shortcut Learning of Large Language Models in Natural Language UnderstandingCommunications of the ACM (CACM), 2022 Mengnan Du Fengxiang He Na Zou Dacheng Tao Helen Zhou KELM OffRL 342 108 0 25 Aug 2022
Outliers Dimensions that Disrupt Transformers Are Driven by FrequencyConference on Empirical Methods in Natural Language Processing (EMNLP), 2022 Giovanni Puccetti Anna Rogers Aleksandr Drozd F. Dell’Orletta 459 54 0 23 May 2022
IsoScore: Measuring the Uniformity of Embedding Space Utilization William Rudman Nate Gillman T. Rayne Carsten Eickhoff 193 34 0 16 Aug 2021