XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization

24 March 2020

Graham Neubig

Papers citing "XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization"

50 / 659 papers shown

Title
Geographic Adaptation of Pretrained Language Models Valentin Hofmann Goran Glavavs Nikola Ljubevsić J. Pierrehumbert Hinrich Schütze VLM 21 16 0 16 Mar 2022
Cross-Lingual Ability of Multilingual Masked Language Models: A Study of Language Structure Yuan Chai Yaobo Liang Nan Duan LRM 27 21 0 16 Mar 2022
MCoNaLa: A Benchmark for Code Generation from Multiple Natural Languages Zhiruo Wang Grace Cuenca Shuyan Zhou Frank F. Xu Graham Neubig 19 49 0 16 Mar 2022
Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction Kuan-Hao Huang I-Hung Hsu Premkumar Natarajan Kai-Wei Chang Nanyun Peng 33 65 0 15 Mar 2022
FairLex: A Multilingual Benchmark for Evaluating Fairness in Legal Text Processing Ilias Chalkidis Tommaso Pasini Shenmin Zhang Letizia Tomada Sebastian Felix Schwemer Anders Søgaard AILaw 32 54 0 14 Mar 2022
Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languages Vaidehi Patil Partha P. Talukdar Sunita Sarawagi 8 21 0 03 Mar 2022
Mukayese: Turkish NLP Strikes Back Ali Safaya Emirhan Kurtulucs Arda Goktougan Deniz Yuret 28 22 0 02 Mar 2022
Large-Scale Hate Speech Detection with Cross-Domain Transfer Cagri Toraman Furkan Şahinuç E. Yilmaz 24 58 0 02 Mar 2022
CINO: A Chinese Minority Pre-trained Language Model Ziqing Yang Zihang Xu Yiming Cui Baoxin Wang Min-Bin Lin Dayong Wu Zhigang Chen 21 25 0 28 Feb 2022
Multi-Level Contrastive Learning for Cross-Lingual Alignment Beiduo Chen Wu Guo Bin Gu Quan Liu Yongchao Wang 18 5 0 26 Feb 2022
Toward More Meaningful Resources for Lower-resourced Languages Constantine Lignos Nolan Holley Chester Palen-Michel Jonne Saleva 11 17 0 24 Feb 2022
NoisyTune: A Little Noise Can Help You Finetune Pretrained Language Models Better Chuhan Wu Fangzhao Wu Tao Qi Yongfeng Huang Xing Xie 25 58 0 24 Feb 2022
Delving Deeper into Cross-lingual Visual Question Answering Chen Cecilia Liu Jonas Pfeiffer Anna Korhonen Ivan Vulić Iryna Gurevych 26 8 0 15 Feb 2022
Slovene SuperGLUE Benchmark: Translation and Evaluation Aleš Žagar Marko Robnik-Šikonja 15 10 0 10 Feb 2022
What are the best systems? New perspectives on NLP Benchmarking Pierre Colombo Nathan Noiry Ekhine Irurozki Stéphan Clémençon 27 28 0 08 Feb 2022
mSLAM: Massively multilingual joint pre-training for speech and text Ankur Bapna Colin Cherry Yu Zhang Ye Jia Melvin Johnson Yong Cheng Simran Khanuja Jason Riesa Alexis Conneau VLM 19 111 0 03 Feb 2022
Constrained Density Matching and Modeling for Cross-lingual Alignment of Contextualized Representations Wei-Ye Zhao Steffen Eger 21 1 0 31 Jan 2022
Cross-Lingual Dialogue Dataset Creation via Outline-Based Generation Olga Majewska E. Razumovskaia E. Ponti Ivan Vulić Anna Korhonen 30 28 0 31 Jan 2022
IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages Emanuele Bugliarello Fangyu Liu Jonas Pfeiffer Siva Reddy Desmond Elliott E. Ponti Ivan Vulić MLLM VLM ELM 40 62 0 27 Jan 2022
HC4: A New Suite of Test Collections for Ad Hoc CLIR Dawn J Lawrie J. Mayfield Douglas W. Oard Eugene Yang VLM AILaw 36 30 0 24 Jan 2022
Leaf: Multiple-Choice Question Generation Kristiyan Vachev Momchil Hardalov Georgi Karadzhov Georgi Georgiev Ivan Koychev Preslav Nakov AI4Ed 25 21 0 22 Jan 2022
Ensemble Transformer for Efficient and Accurate Ranking Tasks: an Application to Question Answering Systems Yoshitomo Matsubara Luca Soldaini Eric Lind Alessandro Moschitti 21 6 0 15 Jan 2022
A Warm Start and a Clean Crawled Corpus -- A Recipe for Good Language Models Vésteinn Snæbjarnarson Haukur Barri Símonarson Pétur Orri Ragnarsson Svanhvít Lilja Ingólfsdóttir H. Jónsson Vilhjálmur Þorsteinsson H. Einarsson 16 26 0 14 Jan 2022
Hypers at ComMA@ICON: Modelling Aggressiveness, Gender Bias and Communal Bias Identification Sean Benhur Roshan Nayak Kanchana Sivanraju Adeep Hande S. Navaneethakrishnan R. Priyadharshini Bharathi Raja Chakravarthi6 27 1 0 31 Dec 2021
Do Multi-Lingual Pre-trained Language Models Reveal Consistent Token Attributions in Different Languages? Junxiang Wang Xuchao Zhang Bo Zong Yanchi Liu Wei Cheng Jingchao Ni Haifeng Chen Liang Zhao AAML 22 0 0 23 Dec 2021
Semantics-Recovering Decompilation through Neural Machine Translation Ruigang Liang Ying Cao Peiwei Hu Jinwen He Kai Chen 14 4 0 22 Dec 2021
On Cross-Lingual Retrieval with Multilingual Text Encoders Robert Litschko Ivan Vulić Simone Paolo Ponzetto Goran Glavavs 11 38 0 21 Dec 2021
LMTurk: Few-Shot Learners as Crowdsourcing Workers in a Language-Model-as-a-Service Framework Mengjie Zhao Fei Mi Yasheng Wang Minglei Li Xin Jiang Qun Liu Hinrich Schütze RALM 33 11 0 14 Dec 2021
Extending the WILDS Benchmark for Unsupervised Adaptation Shiori Sagawa Pang Wei Koh Tony Lee Irena Gao Sang Michael Xie ... Kate Saenko Tatsunori Hashimoto Sergey Levine Chelsea Finn Percy Liang OOD 16 97 0 09 Dec 2021
Question Answering Survey: Directions, Challenges, Datasets, Evaluation Matrices Hariom A. Pandya Brijesh S. Bhatt 38 27 0 07 Dec 2021
Quantifying Adaptability in Pre-trained Language Models with 500 Tasks Belinda Z. Li Jane A. Yu Madian Khabsa Luke Zettlemoyer A. Halevy Jacob Andreas ELM 14 16 0 06 Dec 2021
How not to Lie with a Benchmark: Rearranging NLP Leaderboards Tatiana Shavrina Valentin Malykh ALM ELM 416 10 0 02 Dec 2021
Enhancing Multilingual Language Model with Massive Multilingual Knowledge Triples Linlin Liu Xin Li Ruidan He Lidong Bing Shafiq R. Joty Luo Si KELM 35 18 0 22 Nov 2021
ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning V. Aribandi Yi Tay Tal Schuster J. Rao H. Zheng ... Jianmo Ni Jai Gupta Kai Hui Sebastian Ruder Donald Metzler MoE 15 213 0 22 Nov 2021
Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learning Baijiong Lin Feiyang Ye Yu Zhang Ivor W. Tsang 31 93 0 20 Nov 2021
XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale Arun Babu Changhan Wang Andros Tjandra Kushal Lakhotia Qiantong Xu ... Yatharth Saraf J. Pino Alexei Baevski Alexis Conneau Michael Auli SSL 19 656 0 17 Nov 2021
Cross-language Information Retrieval P. Galuscáková Douglas W. Oard Suraj Nair 18 0 0 10 Nov 2021
Cross-lingual Adaption Model-Agnostic Meta-Learning for Natural Language Understanding Qianying Liu Fei Cheng Sadao Kurohashi 9 1 0 10 Nov 2021
LMdiff: A Visual Diff Tool to Compare Language Models Hendrik Strobelt Benjamin Hoover Arvind Satyanarayan Sebastian Gehrmann VLM 29 19 0 02 Nov 2021
Adapting to the Long Tail: A Meta-Analysis of Transfer Learning Research for Language Understanding Tasks Aakanksha Naik J. Lehman Carolyn Rose 32 7 0 02 Nov 2021
When is BERT Multilingual? Isolating Crucial Ingredients for Cross-lingual Transfer A. Deshpande Partha P. Talukdar Karthik Narasimhan 17 38 0 27 Oct 2021
Evaluating the Evaluation Metrics for Style Transfer: A Case Study in Multilingual Formality Transfer Eleftheria Briakou Sweta Agrawal Joel R. Tetreault Marine Carpuat 23 30 0 20 Oct 2021
SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Training Ankur Bapna Yu-An Chung Na Wu Anmol Gulati Ye Jia J. Clark Melvin Johnson Jason Riesa Alexis Conneau Yu Zhang VLM 51 94 0 20 Oct 2021
Multilingual Domain Adaptation for NMT: Decoupling Language and Domain Information with Adapters Asa Cooper Stickland Alexandre Berard Vassilina Nikoulina AI4CE 11 28 0 18 Oct 2021
Deep Transfer Learning & Beyond: Transformer Language Models in Information Systems Research Ross Gruetzemacher D. Paradice 17 30 0 18 Oct 2021
Predicting the Performance of Multilingual NLP Models A. Srinivasan Sunayana Sitaram T. Ganu Sandipan Dandapat Kalika Bali Monojit Choudhury LRM 30 27 0 17 Oct 2021
Towards Making the Most of Multilingual Pretraining for Zero-Shot Neural Machine Translation Guanhua Chen Shuming Ma Yun-Nung Chen Dongdong Zhang Jia-Yu Pan Wenping Wang Furu Wei LRM 23 14 0 16 Oct 2021
EncT5: A Framework for Fine-tuning T5 as Non-autoregressive Models Frederick Liu T. Huang Shihang Lyu Siamak Shakeri Hongkun Yu Jing Li 31 8 0 16 Oct 2021
Cross-Lingual Fine-Grained Entity Typing N. Selvaraj Yasumasa Onoe Greg Durrett 6 2 0 15 Oct 2021
OpenHands: Making Sign Language Recognition Accessible with Pose-based Pretrained Models across Languages Prem Selvaraj Gokul Nc Pratyush Kumar Mitesh Khapra VLM SLR 48 53 0 12 Oct 2021