First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT

26 January 2021

Papers citing "First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT"

49 / 49 papers shown

Title
High-Dimensional Interlingual Representations of Large Language Models Bryan Wilie Samuel Cahyawijaya Junxian He Pascale Fung 52 0 0 14 Mar 2025
Language Models' Factuality Depends on the Language of Inquiry Tushar Aggarwal Kumar Tanmay Ayush Agrawal Kumar Ayush Hamid Palangi Paul Pu Liang HILM KELM 68 0 0 25 Feb 2025
Beyond Literal Token Overlap: Token Alignability for Multilinguality Katharina Hämmerl Tomasz Limisiewicz Jindrich Libovický Alexander M. Fraser 43 0 0 10 Feb 2025
Layer by Layer: Uncovering Where Multi-Task Learning Happens in Instruction-Tuned Large Language Models Zheng Zhao Yftah Ziser Shay B. Cohen 20 0 0 25 Oct 2024
The Same But Different: Structural Similarities and Differences in Multilingual Language Modeling Ruochen Zhang Qinan Yu Matianyu Zang Carsten Eickhoff Ellie Pavlick 43 1 0 11 Oct 2024
MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment Amir Hossein Kargaran Ali Modarressi Nafiseh Nikeghbal Jana Diesner François Yvon Hinrich Schütze ELM 44 3 0 08 Oct 2024
Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models Lucas Bandarkar Benjamin Muller Pritish Yuvraj Rui Hou Nayan Singhal Hongjiang Lv Bing-Quan Liu KELM LRM MoMe 33 2 0 02 Oct 2024
Probing the Emergence of Cross-lingual Alignment during LLM Training Hetong Wang Pasquale Minervini E. Ponti 20 7 0 19 Jun 2024
Understanding the role of FFNs in driving multilingual behaviour in LLMs Sunit Bhattacharya Ondrej Bojar 21 2 0 22 Apr 2024
Tracing the Roots of Facts in Multilingual Language Models: Independent, Shared, and Transferred Knowledge Xin Zhao Naoki Yoshinaga Daisuke Oba KELM HILM 22 10 0 08 Mar 2024
Analysis of Multi-Source Language Training in Cross-Lingual Transfer Seong Hoon Lim Taejun Yun Jinhyeon Kim Jihun Choi Taeuk Kim 46 2 0 21 Feb 2024
The Hidden Space of Transformer Language Adapters Jesujoba Oluwadara Alabi Marius Mosbach Matan Eyal Dietrich Klakow Mor Geva 48 7 1 20 Feb 2024
Do Llamas Work in English? On the Latent Language of Multilingual Transformers Chris Wendler V. Veselovsky Giovanni Monea Robert West 56 95 0 16 Feb 2024
Analyzing the Evaluation of Cross-Lingual Knowledge Transfer in Multilingual Language Models Sara Rajaee Christof Monz 17 3 0 03 Feb 2024
Discovering Low-rank Subspaces for Language-agnostic Multilingual Representations Zhihui Xie Handong Zhao Tong Yu Shuai Li 26 13 0 11 Jan 2024
MELA: Multilingual Evaluation of Linguistic Acceptability Ziyin Zhang Yikang Liu Wei Huang Junyu Mao Rui Wang Hai Hu 22 3 0 15 Nov 2023
A Joint Matrix Factorization Analysis of Multilingual Representations Zheng Zhao Yftah Ziser Bonnie Webber Shay B. Cohen 16 2 0 24 Oct 2023
Are Structural Concepts Universal in Transformer Language Models? Towards Interpretable Cross-Lingual Generalization Ningyu Xu Qi Zhang Jingting Ye Menghan Zhang Xuanjing Huang 38 4 0 19 Oct 2023
Comparing Styles across Languages: A Cross-Cultural Exploration of Politeness Shreya Havaldar Matthew Pressimone Eric Wong Lyle Ungar 50 2 0 11 Oct 2023
Few-Shot Spoken Language Understanding via Joint Speech-Text Models Chung-Ming Chien Mingjiamei Zhang Ju-Chieh Chou Karen Livescu 26 3 0 09 Oct 2023
The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants Lucas Bandarkar Davis Liang Benjamin Muller Mikel Artetxe Satya Narayan Shukla Don Husa Naman Goyal Abhinandan Krishnan Luke Zettlemoyer Madian Khabsa 28 128 0 31 Aug 2023
Differential Privacy, Linguistic Fairness, and Training Data Influence: Impossibility and Possibility Theorems for Multilingual Language Models Phillip Rust Anders Søgaard 22 3 0 17 Aug 2023
Gradient Sparsification For Masked Fine-Tuning of Transformers J. Ó. Neill Sourav Dutta 14 0 0 19 Jul 2023
CoLaDa: A Collaborative Label Denoising Framework for Cross-lingual Named Entity Recognition Tingting Ma Qianhui Wu Huiqiang Jiang Börje F. Karlsson T. Zhao Chin-Yew Lin 21 4 0 24 May 2023
mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Models Peiqin Lin Chengzhi Hu Zheyu Zhang André F. T. Martins Hinrich Schütze 22 1 0 23 May 2023
How do languages influence each other? Studying cross-lingual data sharing during LM fine-tuning Rochelle Choenni Dan Garrette Ekaterina Shutova 25 15 0 22 May 2023
Measuring Cross-Lingual Transferability of Multilingual Transformers on Sentence Classification Zewen Chi Heyan Huang Xian-Ling Mao 31 0 0 15 May 2023
Identifying the Correlation Between Language Distance and Cross-Lingual Transfer in a Multilingual Representation Space Fred Philippy Siwen Guo Shohreh Haddadan 33 7 0 03 May 2023
ContraSim -- A Similarity Measure Based on Contrastive Learning Adir Rahamim Yonatan Belinkov SSL 17 2 0 29 Mar 2023
In What Languages are Generative Language Models the Most Formal? Analyzing Formality Distribution across Languages Asim Ersoy Gerson Vizcarra T. Mayeesha Benjamin Muller 19 2 0 23 Feb 2023
Cross-Linguistic Syntactic Difference in Multilingual BERT: How Good is It and How Does It Affect Transfer? Ningyu Xu Tao Gui Ruotian Ma Qi Zhang Jingting Ye Menghan Zhang Xuanjing Huang 20 13 0 21 Dec 2022
WIDER & CLOSER: Mixture of Short-channel Distillers for Zero-shot Cross-lingual Named Entity Recognition Jun-Yu Ma Beiduo Chen Jia-Chen Gu Zhen-Hua Ling Wu Guo Quan Liu Zhigang Chen Cong Liu 29 10 0 07 Dec 2022
Cross-lingual Similarity of Multilingual Representations Revisited Maksym Del Mark Fishel 8 3 0 04 Dec 2022
Languages You Know Influence Those You Learn: Impact of Language Characteristics on Multi-Lingual Text-to-Text Transfer Benjamin Muller Deepanshu Gupta Siddharth Patwardhan J. Fauconnier David Vandyke Sachin Agarwal 22 5 0 04 Dec 2022
A Commonsense-Infused Language-Agnostic Learning Framework for Enhancing Prediction of Political Polarity in Multilingual News Headlines Swati Swati Adrian Mladenic Grobelnik Dunja Mladenić M. Grobelnik 14 3 0 01 Dec 2022
Discovering Language-neutral Sub-networks in Multilingual Language Models Negar Foroutan Mohammadreza Banaei R. Lebret Antoine Bosselut Karl Aberer LRM 39 25 0 25 May 2022
Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Models Terra Blevins Hila Gonen Luke Zettlemoyer LRM 54 26 0 24 May 2022
OneAligner: Zero-shot Cross-lingual Transfer with One Rich-Resource Language Pair for Low-Resource Sentence Retrieval Tong Niu Kazuma Hashimoto Yingbo Zhou Caiming Xiong VLM 25 5 0 17 May 2022
Feature Aggregation in Zero-Shot Cross-Lingual Transfer Using Multilingual BERT Beiduo Chen Wu Guo Quan Liu Kun Tao 16 1 0 17 May 2022
Combining Static and Contextualised Multilingual Embeddings Katharina Hämmerl Jindrich Libovický Alexander M. Fraser 20 10 0 17 Mar 2022
Cross-Lingual Ability of Multilingual Masked Language Models: A Study of Language Structure Yuan Chai Yaobo Liang Nan Duan LRM 17 21 0 16 Mar 2022
Multi-Level Contrastive Learning for Cross-Lingual Alignment Beiduo Chen Wu Guo Bin Gu Quan Liu Yongchao Wang 13 5 0 26 Feb 2022
Does Transliteration Help Multilingual Language Modeling? Ibraheem Muhammad Moosa Mahmud Elahi Akhter Ashfia Binte Habib 32 11 0 29 Jan 2022
Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios? Arij Riabi Benoît Sagot Djamé Seddah 26 15 0 26 Oct 2021
Multilingual Counter Narrative Type Classification Yi-Ling Chung Marco Guerini Rodrigo Agerri 75 15 0 28 Sep 2021
Locating Language-Specific Information in Contextualized Embeddings Sheng Liang Philipp Dufter Hinrich Schütze 12 7 0 16 Sep 2021
Similarity of Sentence Representations in Multilingual LMs: Resolving Conflicting Literature and Case Study of Baltic Languages Maksym Del Mark Fishel 9 4 0 02 Sep 2021
Do Multilingual Neural Machine Translation Models Contain Language Pair Specific Attention Heads? Zae Myung Kim Laurent Besacier Vassilina Nikoulina D. Schwab MILM 47 7 0 31 May 2021
What you can cram into a single vector: Probing sentence embeddings for linguistic properties Alexis Conneau Germán Kruszewski Guillaume Lample Loïc Barrault Marco Baroni 199 879 0 03 May 2018