First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT

Conference of the European Chapter of the Association for Computational Linguistics (EACL), 2021

26 January 2021

Papers citing "First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT"

50 / 53 papers shown

Title
Model-Based Ranking of Source Languages for Zero-Shot Cross-Lingual Transfer Abteen Ebrahimi Adam Wiemerslage Katharina von der Wense LRM 159 0 0 03 Oct 2025
Safe and Efficient In-Context Learning via Risk Control Andrea Wynn Metod Jazbec Charith Peris Rinat Khaziev Anqi Liu Daniel Khashabi Eric T. Nalisnick 100 0 0 02 Oct 2025
The Translation Barrier Hypothesis: Multilingual Generation with Large Language Models Suffers from Implicit Translation Failure Niyati Bafna Tianjian Li Kenton W. Murray David R. Mortensen David Yarowsky Hale Sirin Daniel Khashabi LRM 138 4 0 28 Jun 2025
Large Language Models as Psychological Simulators: A Methodological Guide Zhicheng Lin LLMAG 207 2 0 20 Jun 2025
Paths Not Taken: Understanding and Mending the Multilingual Factual Recall Pipeline Meng Lu Ruochen Zhang Carsten Eickhoff Ellie Pavlick HILM KELM LRM 285 6 0 26 May 2025
The Unreasonable Effectiveness of Model Merging for Cross-Lingual Transfer in LLMs Lucas Bandarkar Nanyun Peng MoMe LRM 299 1 0 23 May 2025
High-Dimensional Interlingual Representations of Large Language Models Bryan Wilie Samuel Cahyawijaya Junxian He Pascale Fung 491 0 0 14 Mar 2025
Language Models' Factuality Depends on the Language of Inquiry Tushar Aggarwal Kumar Tanmay Ayush Agrawal Kumar Ayush Hamid Palangi Paul Pu Liang HILM KELM 289 7 0 25 Feb 2025
Beyond Literal Token Overlap: Token Alignability for MultilingualityNorth American Chapter of the Association for Computational Linguistics (NAACL), 2025 Katharina Hämmerl Tomasz Limisiewicz Jindrich Libovický Kangyang Luo 154 3 0 10 Feb 2025
Layer by Layer: Uncovering Where Multi-Task Learning Happens in Instruction-Tuned Large Language ModelsConference on Empirical Methods in Natural Language Processing (EMNLP), 2024 Zheng Zhao Yftah Ziser Shay B. Cohen 167 7 0 25 Oct 2024
The Same But Different: Structural Similarities and Differences in Multilingual Language ModelingInternational Conference on Learning Representations (ICLR), 2024 Ruochen Zhang Qinan Yu Matianyu Zang Carsten Eickhoff Ellie Pavlick 247 14 0 11 Oct 2024
MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual AlignmentAnnual Meeting of the Association for Computational Linguistics (ACL), 2024 Amir Hossein Kargaran Ali Modarressi Nafiseh Nikeghbal Jana Diesner François Yvon Hinrich Schütze ELM 312 16 0 08 Oct 2024
Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language ModelsInternational Conference on Learning Representations (ICLR), 2024 Lucas Bandarkar Benjamin Muller Pritish Yuvraj Rui Hou Nayan Singhal Hongjiang Lv Bing-Quan Liu KELM LRM MoMe 427 12 0 02 Oct 2024
Probing the Emergence of Cross-lingual Alignment during LLM TrainingAnnual Meeting of the Association for Computational Linguistics (ACL), 2024 Hetong Wang Pasquale Minervini Edoardo Ponti 330 27 0 19 Jun 2024
Understanding the role of FFNs in driving multilingual behaviour in LLMs Sunit Bhattacharya Ondrej Bojar 137 4 0 22 Apr 2024
Tracing the Roots of Facts in Multilingual Language Models: Independent, Shared, and Transferred KnowledgeConference of the European Chapter of the Association for Computational Linguistics (EACL), 2024 Xin Zhao Naoki Yoshinaga Daisuke Oba KELM HILM 152 16 0 08 Mar 2024
Analysis of Multi-Source Language Training in Cross-Lingual Transfer Seong Hoon Lim Taejun Yun Jinhyeon Kim Jihun Choi Taeuk Kim 213 5 0 21 Feb 2024
The Hidden Space of Transformer Language Adapters Jesujoba Oluwadara Alabi Marius Mosbach Matan Eyal Dietrich Klakow Mor Geva 327 15 1 20 Feb 2024
Do Llamas Work in English? On the Latent Language of Multilingual Transformers Chris Wendler V. Veselovsky Giovanni Monea Robert West 529 212 0 16 Feb 2024
Analyzing the Evaluation of Cross-Lingual Knowledge Transfer in Multilingual Language Models Sara Rajaee Christof Monz 230 10 0 03 Feb 2024
Discovering Low-rank Subspaces for Language-agnostic Multilingual RepresentationsConference on Empirical Methods in Natural Language Processing (EMNLP), 2024 Zhihui Xie Handong Zhao Tong Yu Shuai Li 191 16 0 11 Jan 2024
MELA: Multilingual Evaluation of Linguistic AcceptabilityAnnual Meeting of the Association for Computational Linguistics (ACL), 2023 Ziyin Zhang Yikang Liu Wei-Ping Huang Junyu Mao Rui Wang Hai Hu 264 15 0 15 Nov 2023
A Joint Matrix Factorization Analysis of Multilingual RepresentationsConference on Empirical Methods in Natural Language Processing (EMNLP), 2023 Zheng Zhao Yftah Ziser Bonnie Webber Shay B. Cohen 220 4 0 24 Oct 2023
Are Structural Concepts Universal in Transformer Language Models? Towards Interpretable Cross-Lingual Generalization Ningyu Xu Tao Gui Jingting Ye Menghan Zhang Xuanjing Huang 279 6 0 19 Oct 2023
Comparing Styles across Languages: A Cross-Cultural Exploration of PolitenessConference on Empirical Methods in Natural Language Processing (EMNLP), 2023 Shreya Havaldar Matthew Pressimone Eric Wong Lyle Ungar 436 2 0 11 Oct 2023
Few-Shot Spoken Language Understanding via Joint Speech-Text ModelsAutomatic Speech Recognition & Understanding (ASRU), 2023 Chung-Ming Chien Mingjiamei Zhang Ju-Chieh Chou Karen Livescu 214 6 0 09 Oct 2023
The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language VariantsAnnual Meeting of the Association for Computational Linguistics (ACL), 2023 Lucas Bandarkar Davis Liang Benjamin Muller Mikel Artetxe Satya Narayan Shukla Don Husa Naman Goyal Abhinandan Krishnan Luke Zettlemoyer Madian Khabsa 324 230 0 31 Aug 2023
Differential Privacy, Linguistic Fairness, and Training Data Influence: Impossibility and Possibility Theorems for Multilingual Language ModelsInternational Conference on Machine Learning (ICML), 2023 Phillip Rust Anders Søgaard 158 6 0 17 Aug 2023
Gradient Sparsification For Masked Fine-Tuning of TransformersIEEE International Joint Conference on Neural Network (IJCNN), 2023 J. Ó. Neill Sourav Dutta 144 0 0 19 Jul 2023
CoLaDa: A Collaborative Label Denoising Framework for Cross-lingual Named Entity RecognitionAnnual Meeting of the Association for Computational Linguistics (ACL), 2023 Tingting Ma Qianhui Wu Huiqiang Jiang Börje F. Karlsson Tiejun Zhao Chin-Yew Lin 233 8 0 24 May 2023
mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language ModelsFindings (Findings), 2023 Peiqin Lin Chengzhi Hu Zheyu Zhang André F. T. Martins Hinrich Schütze 203 1 0 23 May 2023
How do languages influence each other? Studying cross-lingual data sharing during LM fine-tuningConference on Empirical Methods in Natural Language Processing (EMNLP), 2023 Rochelle Choenni Dan Garrette Ekaterina Shutova 288 19 0 22 May 2023
Measuring Cross-Lingual Transferability of Multilingual Transformers on Sentence Classification Zewen Chi Heyan Huang Xian-Ling Mao 239 0 0 15 May 2023
Identifying the Correlation Between Language Distance and Cross-Lingual Transfer in a Multilingual Representation Space Fred Philippy Siwen Guo Shohreh Haddadan 144 10 0 03 May 2023
ContraSim -- A Similarity Measure Based on Contrastive LearningNorth American Chapter of the Association for Computational Linguistics (NAACL), 2023 Adir Rahamim Yonatan Belinkov SSL 210 4 0 29 Mar 2023
In What Languages are Generative Language Models the Most Formal? Analyzing Formality Distribution across LanguagesConference on Empirical Methods in Natural Language Processing (EMNLP), 2023 Asim Ersoy Gerson Vizcarra T. Mayeesha Benjamin Muller 206 4 0 23 Feb 2023
Cross-Linguistic Syntactic Difference in Multilingual BERT: How Good is It and How Does It Affect Transfer?Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022 Ningyu Xu Tao Gui Ruotian Ma Tao Gui Jingting Ye Menghan Zhang Xuanjing Huang 209 14 0 21 Dec 2022
WIDER & CLOSER: Mixture of Short-channel Distillers for Zero-shot Cross-lingual Named Entity RecognitionConference on Empirical Methods in Natural Language Processing (EMNLP), 2022 Jun-Yu Ma Beiduo Chen Jia-Chen Gu Zhen-Hua Ling Wu Guo Quan Liu Zhigang Chen Cong Liu 176 12 0 07 Dec 2022
Cross-lingual Similarity of Multilingual Representations Revisited Maksym Del Mark Fishel 118 5 0 04 Dec 2022
Languages You Know Influence Those You Learn: Impact of Language Characteristics on Multi-Lingual Text-to-Text Transfer Benjamin Muller Deepanshu Gupta Siddharth Patwardhan J. Fauconnier David Vandyke Sachin Agarwal 213 6 0 04 Dec 2022
A Commonsense-Infused Language-Agnostic Learning Framework for Enhancing Prediction of Political Polarity in Multilingual News HeadlinesKnowledge-Based Systems (KBS), 2022 Swati Swati Adrian Mladenic Grobelnik Dunja Mladenić M. Grobelnik 197 4 0 01 Dec 2022
Discovering Language-neutral Sub-networks in Multilingual Language ModelsConference on Empirical Methods in Natural Language Processing (EMNLP), 2022 Negar Foroutan Mohammadreza Banaei R. Lebret Antoine Bosselut Karl Aberer LRM 232 27 0 25 May 2022
Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language ModelsConference on Empirical Methods in Natural Language Processing (EMNLP), 2022 Terra Blevins Hila Gonen Luke Zettlemoyer LRM 219 36 0 24 May 2022
OneAligner: Zero-shot Cross-lingual Transfer with One Rich-Resource Language Pair for Low-Resource Sentence RetrievalFindings (Findings), 2022 Tong Niu Kazuma Hashimoto Yingbo Zhou Caiming Xiong VLM 107 7 0 17 May 2022
Feature Aggregation in Zero-Shot Cross-Lingual Transfer Using Multilingual BERTInternational Conference on Pattern Recognition (ICPR), 2022 Beiduo Chen Wu Guo Quan Liu Kun Tao 200 3 0 17 May 2022
Combining Static and Contextualised Multilingual EmbeddingsFindings (Findings), 2022 Katharina Hämmerl Jindrich Libovický Kangyang Luo 209 16 0 17 Mar 2022
Cross-Lingual Ability of Multilingual Masked Language Models: A Study of Language StructureAnnual Meeting of the Association for Computational Linguistics (ACL), 2022 Yuan Chai Yaobo Liang Nan Duan LRM 158 26 0 16 Mar 2022
Multi-Level Contrastive Learning for Cross-Lingual AlignmentIEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2022 Beiduo Chen Wu Guo Bin Gu Quan Liu Yongchao Wang 226 6 0 26 Feb 2022
Does Transliteration Help Multilingual Language Modeling?Findings (Findings), 2022 Ibraheem Muhammad Moosa Mahmud Elahi Akhter Ashfia Binte Habib 301 14 0 29 Jan 2022
Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios? Arij Riabi Benoît Sagot Djamé Seddah 239 16 0 26 Oct 2021