A Primer on Pretrained Multilingual Language Models

A Primer on Pretrained Multilingual Language Models

1 July 2021

Sumanth Doddapaneni

Mitesh M. Khapra

Anoop Kunchukuttan

Papers citing "A Primer on Pretrained Multilingual Language Models"

19 / 19 papers shown

Title
Better To Ask in English? Evaluating Factual Accuracy of Multilingual LLMs in English and Low-Resource Languages Pritika Rohera Chaitrali Ginimav Gayatri Sawant Raviraj Joshi 21 0 0 28 Apr 2025
Can xLLMs Understand the Structure of Dialog? Exploring Multilingual Response Generation in Complex Scenarios Zhongtian Hu Yiwen Cui Ronghan Li Meng Zhao Lifang Wang 29 0 0 20 Jan 2025
The Ghanaian NLP Landscape: A First Look Sheriff Issaka Zhaoyi Zhang Mihir Heda Keyi Wang Yinka Ajibola Ryan DeMar Xuefeng Du 23 1 0 10 May 2024
Multilingual Brain Surgeon: Large Language Models Can be Compressed Leaving No Language Behind Hongchuan Zeng Hongshen Xu Lu Chen Kai Yu 32 4 0 06 Apr 2024
Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance Jiasheng Ye Peiju Liu Tianxiang Sun Yunhua Zhou Jun Zhan Xipeng Qiu 26 58 0 25 Mar 2024
Testing the Predictions of Surprisal Theory in 11 Languages Ethan Gotlieb Wilcox Tiago Pimentel Clara Meister Ryan Cotterell R. Levy LRM 24 62 0 07 Jul 2023
Pretraining with Artificial Language: Studying Transferable Knowledge in Language Models Ryokan Ri Yoshimasa Tsuruoka 4 25 0 19 Mar 2022
Word Alignment by Fine-tuning Embeddings on Parallel Corpora Zi-Yi Dou Graham Neubig 90 255 0 20 Jan 2021
Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processing Minh Nguyen Viet Dac Lai Amir Pouran Ben Veyseh Thien Huu Nguyen 41 115 0 09 Jan 2021
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora Ouyang Xuan Shuohuan Wang Chao Pang Yu Sun Hao Tian Hua-Hong Wu Haifeng Wang 51 100 0 31 Dec 2020
How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models Phillip Rust Jonas Pfeiffer Ivan Vulić Sebastian Ruder Iryna Gurevych 69 235 0 31 Dec 2020
Orthogonal Language and Task Adapters in Zero-Shot Cross-Lingual Transfer M. Vidoni Ivan Vulić Goran Glavas 23 27 0 11 Dec 2020
When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models Benjamin Muller Antonis Anastasopoulos Benoît Sagot Djamé Seddah LRM 101 150 0 24 Oct 2020
Rethinking embedding coupling in pre-trained language models Hyung Won Chung Thibault Févry Henry Tsai Melvin Johnson Sebastian Ruder 75 142 0 24 Oct 2020
Improving Multilingual Models with Language-Clustered Vocabularies Hyung Won Chung Dan Garrette Kiat Chuan Tan Jason Riesa VLM 55 56 0 24 Oct 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 224 1,281 0 18 Mar 2020
MLQA: Evaluating Cross-lingual Extractive Question Answering Patrick Lewis Barlas Oğuz Ruty Rinott Sebastian Riedel Holger Schwenk ELM 239 489 0 16 Oct 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Z. Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 716 6,435 0 26 Sep 2016