Exploring Anisotropy and Outliers in Multilingual Language Models for Cross-Lingual Semantic Sentence Similarity

1 June 2023

Papers citing "Exploring Anisotropy and Outliers in Multilingual Language Models for Cross-Lingual Semantic Sentence Similarity"

8 / 8 papers shown

Title
Outlier dimensions favor frequent tokens in language models Iuri Macocco Nora Graichen Gemma Boleda Marco Baroni 55 0 0 27 Mar 2025
MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment Amir Hossein Kargaran Ali Modarressi Nafiseh Nikeghbal Jana Diesner François Yvon Hinrich Schütze ELM 46 3 0 08 Oct 2024
Isotropy, Clusters, and Classifiers Timothee Mickus Stig-Arne Gronroos Joseph Attieh 38 0 0 05 Feb 2024
Anisotropy Is Inherent to Self-Attention in Transformers Nathan Godey Eric Villemonte de la Clergerie Benoît Sagot 15 16 0 22 Jan 2024
Outliers Dimensions that Disrupt Transformers Are Driven by Frequency Giovanni Puccetti Anna Rogers Aleksandr Drozd F. Dell’Orletta 76 42 0 23 May 2022
Pairwise Supervised Contrastive Learning of Sentence Representations Dejiao Zhang Shang-Wen Li Wei Xiao Henghui Zhu Ramesh Nallapati Andrew O. Arnold Bing Xiang SSL 55 50 0 12 Sep 2021
How Does Fine-tuning Affect the Geometry of Embedding Space: A Case Study on Isotropy S. Rajaee Mohammad Taher Pilehvar 76 20 0 10 Sep 2021
All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality William Timkey Marten van Schijndel 213 110 0 09 Sep 2021