Lifting the Curse of Multilinguality by Pre-training Modular Transformers

12 May 2022

Xian Li

Papers citing "Lifting the Curse of Multilinguality by Pre-training Modular Transformers"

50 / 116 papers shown

Title
Investigating the Effect of Parallel Data in the Cross-Lingual Transfer for Vision-Language Encoders Andrei-Alexandru Manea Jindřich Libovický VLM 47 0 0 30 Apr 2025
Llama-3-Nanda-10B-Chat: An Open Generative Large Language Model for Hindi Monojit Choudhury Shivam Chauhan Rocktim Jyoti Das Dhruv Sahnan Xudong Han ... Rituraj Joshi Gurpreet Gosal Avraham Sheinin Natalia Vassilieva Preslav Nakov 18 0 0 08 Apr 2025
Improving Low-Resource Retrieval Effectiveness using Zero-Shot Linguistic Similarity Transfer Andreas Chari Sean MacAvaney Iadh Ounis 34 0 0 28 Mar 2025
Trustworthy Machine Learning via Memorization and the Granular Long-Tail: A Survey on Interactions, Tradeoffs, and Beyond Qiongxiu Li Xiaoyu Luo Yiyi Chen Johannes Bjerva 43 0 0 10 Mar 2025
Linguistic Generalizability of Test-Time Scaling in Mathematical Reasoning Guijin Son Jiwoo Hong Hyunwoo Ko James Thorne LRM 46 5 0 24 Feb 2025
Extracting General-use Transformers for Low-resource Languages via Knowledge Distillation Jan Christian Blaise Cruz Alham Fikri Aji 35 1 0 22 Jan 2025
INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge Angelika Romanou Negar Foroutan Anna Sotnikova Zeming Chen Sree Harsha Nelaturu ... Mike Zhang Imanol Schlag Marzieh Fadaee Sara Hooker Antoine Bosselut ELM 92 5 0 29 Nov 2024
From English-Centric to Effective Bilingual: LLMs with Custom Tokenizers for Underrepresented Languages Artur Kiulian Anton Polishko M. Khandoga Yevhen Kostiuk Guillermo Gabrielli ... Hrishikesh Garud Wendy Wing Yee Mak Dmytro Chaplynskyi Selma Belhadj Amor Grigol Peradze 25 0 0 24 Oct 2024
Exploring Continual Fine-Tuning for Enhancing Language Ability in Large Language Model Divyanshu Aggarwal Sankarshan Damle Navin Goyal Satya Lokam Sunayana Sitaram CLL 15 0 0 21 Oct 2024
Breaking the Manual Annotation Bottleneck: Creating a Comprehensive Legal Case Criticality Dataset through Semi-Automated Labeling Ronja Stern Ken Kawamura Matthias Sturmer Ilias Chalkidis Joel Niklaus AILaw ELM 24 0 0 17 Oct 2024
Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence Shangbin Feng Zifeng Wang Yike Wang Sayna Ebrahimi Hamid Palangi ... Nathalie Rauschmayr Yejin Choi Yulia Tsvetkov Chen-Yu Lee Tomas Pfister MoMe 27 3 0 15 Oct 2024
Efficiently Democratizing Medical LLMs for 50 Languages via a Mixture of Language Family Experts Guorui Zheng Xidong Wang Juhao Liang Nuo Chen Yuping Zheng Benyou Wang MoE 17 5 0 14 Oct 2024
Linguistically-Informed Multilingual Instruction Tuning: Is There an Optimal Set of Languages to Tune? Gürkan Soykan Gözde Gül Şahin 18 0 0 10 Oct 2024
OD-Stega: LLM-Based Near-Imperceptible Steganography via Optimized Distributions Yu-Shin Huang Peter Just Krishna Narayanan Chao Tian 28 2 0 06 Oct 2024
Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models Lucas Bandarkar Benjamin Muller Pritish Yuvraj Rui Hou Nayan Singhal Hongjiang Lv Bing-Quan Liu KELM LRM MoMe 21 2 0 02 Oct 2024
Exploring Intrinsic Language-specific Subspaces in Fine-tuning Multilingual Neural Machine Translation Zhe Cao Zhi Qu Hidetaka Kamigaito Taro Watanabe MoE 25 0 0 08 Sep 2024
Bridging the Language Gap: Enhancing Multilingual Prompt-Based Code Generation in LLMs via Zero-Shot Cross-Lingual Transfer Mingda Li Abhijit Mishra Utkarsh Mujumdar 24 0 0 19 Aug 2024
Modular Sentence Encoders: Separating Language Specialization from Cross-Lingual Alignment Yongxin Huang Kexin Wang Goran Glavavs Iryna Gurevych 39 0 0 20 Jul 2024
Fixed and Adaptive Simultaneous Machine Translation Strategies Using Adapters Abderrahmane Issam Yusuf Can Semerci Jan Scholtes Gerasimos Spanakis 24 0 0 18 Jul 2024
On the Limitations of Compute Thresholds as a Governance Strategy Sara Hooker 37 14 0 08 Jul 2024
Soft Language Prompts for Language Transfer Ivan Vykopal Simon Ostermann Marián Simko AAML 24 1 0 02 Jul 2024
Adapting Multilingual LLMs to Low-Resource Languages with Knowledge Graphs via Adapters Daniil Gurgurov Mareike Hartmann Simon Ostermann 34 6 0 01 Jul 2024
Segment Any Text: A Universal Approach for Robust, Efficient and Adaptable Sentence Segmentation Markus Frohmann Igor Sterner Ivan Vulić Benjamin Minixhofer Markus Schedl VLM 36 11 0 24 Jun 2024
Multilingual Large Language Models and Curse of Multilinguality Daniil Gurgurov Tanja Bäumel Tatiana Anikina 63 4 0 15 Jun 2024
BertaQA: How Much Do Language Models Know About Local Culture? Julen Etxaniz Gorka Azkune A. Soroa Oier López de Lacalle Mikel Artetxe 33 6 0 11 Jun 2024
Aya 23: Open Weight Releases to Further Multilingual Progress Viraat Aryabumi John Dang Dwarak Talupuru Saurabh Dash David Cairuz ... Aidan N. Gomez Phil Blunsom Marzieh Fadaee A. Ustun Sara Hooker OSLM 39 72 0 23 May 2024
Fine-tuning the SwissBERT Encoder Model for Embedding Sentences and Documents Juri Grosjean Jannis Vamvas 21 1 0 13 May 2024
Efficient Compression of Multitask Multilingual Speech Models Thomas Palmeira Ferraz 33 0 0 02 May 2024
No Train but Gain: Language Arithmetic for training-free Language Adapters enhancement Mateusz Klimaszewski Piotr Andruszkiewicz Alexandra Birch MoMe 29 4 0 24 Apr 2024
Neuron Specialization: Leveraging intrinsic task modularity for multilingual machine translation Shaomu Tan Di Wu Christof Monz MoMe 23 7 0 17 Apr 2024
The Role of Language Imbalance in Cross-lingual Generalisation: Insights from Cloned Language Experiments Anton Schäfer Shauli Ravfogel Thomas Hofmann Tiago Pimentel Imanol Schlag 52 3 0 11 Apr 2024
Willkommens-Merkel, Chaos-Johnson, and Tore-Klose: Modeling the Evaluative Meaning of German Personal Name Compounds Annerose Eichel Tana Deeg André Blessing Milena Belosevic Sabine Arndt-Lappe Sabine Schulte im Walde 24 0 0 05 Apr 2024
Poro 34B and the Blessing of Multilinguality Risto Luukkonen Jonathan Burdge Elaine Zosa Aarne Talman Ville Komulainen Vaino Hatanpaa Peter Sarlin S. Pyysalo AI4CE 36 12 0 02 Apr 2024
AAdaM at SemEval-2024 Task 1: Augmentation and Adaptation for Multilingual Semantic Textual Relatedness Miaoran Zhang Mingyang Wang Jesujoba Oluwadara Alabi Dietrich Klakow VLM 28 4 0 01 Apr 2024
An Efficient Approach for Studying Cross-Lingual Transfer in Multilingual Language Models Fahim Faisal Antonios Anastasopoulos 19 0 0 29 Mar 2024
NaijaHate: Evaluating Hate Speech Detection on Nigerian Twitter Using Representative Data Manuel Tonneau Pedro Vitor Quinta de Castro Karim Lasri I. Farouq Lakshminarayanan Subramanian Victor Orozco-Olvera Samuel Fraiberger 25 9 0 28 Mar 2024
Towards a World-English Language Model for On-Device Virtual Assistants Rricha Jalota Lyan Verwimp Markus Nussbaum-Thom Amr Mousa Arturo Argueta Youssef Oualil 16 0 0 27 Mar 2024
SumTra: A Differentiable Pipeline for Few-Shot Cross-Lingual Summarization Jacob Parnell Inigo Jauregi Unanue Massimo Piccardi 18 2 0 20 Mar 2024
Comparing Explanation Faithfulness between Multilingual and Monolingual Fine-tuned Language Models Zhixue Zhao Nikolaos Aletras 18 3 0 19 Mar 2024
Conditional computation in neural networks: principles and research trends Simone Scardapane Alessandro Baiocchi Alessio Devoto V. Marsocci Pasquale Minervini Jary Pomponi 27 0 0 12 Mar 2024
Few-Shot Cross-Lingual Transfer for Prompting Large Language Models in Low-Resource Languages Christopher Toukmaji LRM 24 0 0 09 Mar 2024
IRCoder: Intermediate Representations Make Language Models Robust Multilingual Code Generators Indraneil Paul Goran Glavas Iryna Gurevych 24 12 0 06 Mar 2024
Multi-Task Contrastive Learning for 8192-Token Bilingual Text Embeddings Isabelle Mohr Markus Krimmel Saba Sturua Mohammad Kalim Akram Andreas Koukounas ... Susana Guzman Bo Wang Maximilian Werk Nan Wang Han Xiao 17 14 0 26 Feb 2024
ColBERT-XM: A Modular Multi-Vector Representation Model for Zero-Shot Multilingual Information Retrieval Antoine Louis V. Saxena Gijs van Dijck Gerasimos Spanakis 27 5 0 23 Feb 2024
Investigating Cultural Alignment of Large Language Models Badr AlKhamissi Muhammad N. ElNokrashy Mai AlKhamissi Mona T. Diab 18 42 0 20 Feb 2024
KMMLU: Measuring Massive Multitask Language Understanding in Korean Guijin Son Hanwool Albert Lee Sungdong Kim Seungone Kim Niklas Muennighoff Taekyoon Choi Cheonbok Park Kang Min Yoo Stella Biderman ALM RALM ELM 30 23 0 18 Feb 2024
Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model A. Ustun Viraat Aryabumi Zheng-Xin Yong Wei-Yin Ko Daniel D'souza ... Shayne Longpre Niklas Muennighoff Marzieh Fadaee Julia Kreutzer Sara Hooker ALM ELM SyDa LRM 27 192 0 12 Feb 2024
The Impact of Language Adapters in Cross-Lingual Transfer for NLU Jenny Kunz Oskar Holmström 20 4 0 31 Jan 2024
Modular Adaptation of Multilingual Encoders to Written Swiss German Dialect Jannis Vamvas Noëmi Aepli Rico Sennrich 22 0 0 25 Jan 2024
Breaking the Curse of Multilinguality with Cross-lingual Expert Language Models Terra Blevins Tomasz Limisiewicz Suchin Gururangan Margaret Li Hila Gonen Noah A. Smith Luke Zettlemoyer 39 22 0 19 Jan 2024