Lifting the Curse of Multilinguality by Pre-training Modular
Transformers

Lifting the Curse of Multilinguality by Pre-training Modular Transformers

12 May 2022

Xi Victoria Lin

Xian Li

Sebastian Riedel

Papers citing "Lifting the Curse of Multilinguality by Pre-training Modular Transformers"

16 / 116 papers shown

Title
Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models Margaret Li Suchin Gururangan Tim Dettmers M. Lewis Tim Althoff Noah A. Smith Luke Zettlemoyer MoMe 18 142 0 05 Aug 2022
Massively Multilingual Lexical Specialization of Multilingual Transformers Tommaso Green Simone Paolo Ponzetto Goran Glavavs 14 0 0 01 Aug 2022
Language Modelling with Pixels Phillip Rust Jonas F. Lotz Emanuele Bugliarello Elizabeth Salesky Miryam de Lhoneux Desmond Elliott VLM 12 46 0 14 Jul 2022
Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset Peter Henderson M. Krass Lucia Zheng Neel Guha Christopher D. Manning Dan Jurafsky Daniel E. Ho AILaw ELM 127 94 0 01 Jul 2022
Cross-lingual Lifelong Learning Meryem M'hamdi Xiang Ren Jonathan May CLL 29 7 0 23 May 2022
Phylogeny-Inspired Adaptation of Multilingual Models to New Languages Fahim Faisal Antonios Anastasopoulos AI4CE LRM 28 26 0 19 May 2022
Subword Mapping and Anchoring across Languages Giorgos Vernikos Andrei Popescu-Belis 52 12 0 09 Sep 2021
Signal Transformer: Complex-valued Attention and Meta-Learning for Signal Recognition Yihong Dong Ying Peng Muqiao Yang Songtao Lu Qingjiang Shi 32 8 0 05 Jun 2021
AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages Abteen Ebrahimi Manuel Mager Arturo Oncevay Vishrav Chaudhary Luis Chiruzzo ... Graham Neubig Alexis Palmer Rolando A. Coto Solano Ngoc Thang Vu Katharina Kann 99 71 0 18 Apr 2021
Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems E. Razumovskaia Goran Glavavs Olga Majewska E. Ponti Anna Korhonen Ivan Vulić 13 28 0 17 Apr 2021
How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models Phillip Rust Jonas Pfeiffer Ivan Vulić Sebastian Ruder Iryna Gurevych 69 235 0 31 Dec 2020
Orthogonal Language and Task Adapters in Zero-Shot Cross-Lingual Transfer M. Vidoni Ivan Vulić Goran Glavas 29 27 0 11 Dec 2020
When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models Benjamin Muller Antonis Anastasopoulos Benoît Sagot Djamé Seddah LRM 106 150 0 24 Oct 2020
Rethinking embedding coupling in pre-trained language models Hyung Won Chung Thibault Févry Henry Tsai Melvin Johnson Sebastian Ruder 84 142 0 24 Oct 2020
Improving Multilingual Models with Language-Clustered Vocabularies Hyung Won Chung Dan Garrette Kiat Chuan Tan Jason Riesa VLM 58 56 0 24 Oct 2020
MLQA: Evaluating Cross-lingual Extractive Question Answering Patrick Lewis Barlas Oğuz Ruty Rinott Sebastian Riedel Holger Schwenk ELM 239 489 0 16 Oct 2019