XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization

24 March 2020

Graham Neubig

Papers citing "XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization"

50 / 659 papers shown

Title
Why Does Zero-Shot Cross-Lingual Generation Fail? An Explanation and a Solution Tianjian Li Kenton W. Murray 27 23 0 27 May 2023
Free Lunch: Robust Cross-Lingual Transfer via Model Checkpoint Averaging Fabian David Schmidt Ivan Vulić Goran Glavavs 8 8 0 26 May 2023
Towards a Common Understanding of Contributing Factors for Cross-Lingual Transfer in Multilingual Language Models: A Review Fred Philippy Siwen Guo Shohreh Haddadan LRM 22 28 0 26 May 2023
Revisiting non-English Text Simplification: A Unified Multilingual Benchmark Michael Joseph Ryan Tarek Naous Wei-ping Xu 24 24 0 25 May 2023
Dolphin: A Challenging and Diverse Benchmark for Arabic NLG El Moatez Billah Nagoudi AbdelRahim Elmadany Ahmed Oumar El-Shangiti Muhammad Abdul-Mageed LM&MA 30 17 0 24 May 2023
BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfer Akari Asai Sneha Kudugunta Xinyan Velocity Yu Terra Blevins Hila Gonen Machel Reid Yulia Tsvetkov Sebastian Ruder Hannaneh Hajishirzi 31 53 0 24 May 2023
GlobalBench: A Benchmark for Global Progress in Natural Language Processing Yueqi Song Catherine Cui Simran Khanuja Pengfei Liu Fahim Faisal ... Alham Fikri Aji Samuel Cahyawijaya Yulia Tsvetkov Antonios Anastasopoulos Graham Neubig 17 7 0 24 May 2023
Having Beer after Prayer? Measuring Cultural Bias in Large Language Models Tarek Naous Michael Joseph Ryan Alan Ritter Wei-ping Xu 24 84 0 23 May 2023
Revisiting Machine Translation for Cross-lingual Classification Mikel Artetxe Vedanuj Goswami Shruti Bhosale Angela Fan Luke Zettlemoyer LRM 40 34 0 23 May 2023
mmT5: Modular Multilingual Pre-Training Solves Source Language Hallucinations Jonas Pfeiffer Francesco Piccinno Massimo Nicosia Xinyi Wang Machel Reid Sebastian Ruder VLM LRM 29 27 0 23 May 2023
$μ$ PLAN: Summarizing using a Content Plan as Cross-Lingual Bridge Fantine Huot Joshua Maynez Chris Alberti Reinald Kim Amplayo Priyanka Agrawal Constanza Fierro Shashi Narayan Mirella Lapata 15 7 0 23 May 2023
Out-of-Distribution Generalization in Text Classification: Past, Present, and Future Linyi Yang Y. Song Xuan Ren Chenyang Lyu Yidong Wang Lingqiao Liu Jindong Wang Jennifer Foster Yue Zhang OOD 20 2 0 23 May 2023
mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Models Peiqin Lin Chengzhi Hu Zheyu Zhang André F. T. Martins Hinrich Schütze 27 1 0 23 May 2023
Understanding and Mitigating Spurious Correlations in Text Classification with Neighborhood Analysis Oscar Chew Hsuan-Tien Lin Kai-Wei Chang Kuan-Hao Huang 30 5 0 23 May 2023
mPMR: A Multilingual Pre-trained Machine Reader at Scale Weiwen Xu Xin Li Wai Lam Lidong Bing LRM 30 5 0 23 May 2023
Detecting and Mitigating Hallucinations in Multilingual Summarisation Yifu Qiu Yftah Ziser Anna Korhonen E. Ponti Shay B. Cohen HILM 49 42 0 23 May 2023
Improving Self-training for Cross-lingual Named Entity Recognition with Contrastive and Prototype Learning Ran Zhou Xin Li Lidong Bing Erik Cambria Chun Miao 35 16 0 23 May 2023
Translation and Fusion Improves Zero-shot Cross-lingual Information Extraction Yang Chen Vedaant Shah Alan Ritter 21 4 0 23 May 2023
Extrapolating Multilingual Understanding Models as Multilingual Generators Bohong Wu Fei Yuan Hai Zhao Lei Li Jingjing Xu AI4CE 20 2 0 22 May 2023
DUMB: A Benchmark for Smart Evaluation of Dutch Models Wietse de Vries Martijn B. Wieling Malvina Nissim ELM ALM MoE 26 6 0 22 May 2023
Is Translation Helpful? An Empirical Analysis of Cross-Lingual Transfer in Low-Resource Dialog Generation Lei Shen Shuai Yu Xiaoyu Shen 14 0 0 21 May 2023
Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages Ayyoob Imani Peiqin Lin Amir Hossein Kargaran Silvia Severini Masoud Jalili Sabet ... Chunlan Ma Helmut Schmid André F. T. Martins François Yvon Hinrich Schütze ALM LRM 29 95 0 20 May 2023
ESCOXLM-R: Multilingual Taxonomy-driven Pre-training for the Job Market Domain Mike Zhang Rob van der Goot Barbara Plank 11 14 0 20 May 2023
XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languages Sebastian Ruder J. Clark Alexander Gutkin Mihir Kale Min Ma ... Dan Garrette R. Ingle Melvin Johnson Dmitry Panteleev Partha P. Talukdar ELM 22 38 0 19 May 2023
Separating form and meaning: Using self-consistency to quantify task understanding across multiple senses Xenia Ohmer Elia Bruni Dieuwke Hupkes LRM 15 13 0 19 May 2023
CCT-Code: Cross-Consistency Training for Multilingual Clone Detection and Code Search Nikita Sorokin Dmitry Abulkhanov Sergey I. Nikolenko Valentin Malykh 21 3 0 19 May 2023
Analyzing and Reducing the Performance Gap in Cross-Lingual Transfer with Fine-tuning Slow and Fast Yiduo Guo Yaobo Liang Dongyan Zhao Bin Liu Du Nan CLL 10 1 0 19 May 2023
Massively Multi-Lingual Event Understanding: Extraction, Visualization, and Search Chris Jenkins Shantanu Agarwal Joel Barry Steven Fincke Elizabeth Boschee 16 5 0 17 May 2023
Towards More Robust NLP System Evaluation: Handling Missing Scores in Benchmarks Anas Himmi Ekhine Irurozki Nathan Noiry Stéphan Clémençon Pierre Colombo 26 5 0 17 May 2023
Variable-length Neural Interlingua Representations for Zero-shot Neural Machine Translation Zhuoyuan Mao Haiyue Song Raj Dabre Chenhui Chu Sadao Kurohashi 27 0 0 17 May 2023
xPQA: Cross-Lingual Product Question Answering across 12 Languages Xiaoyu Shen Akari Asai Bill Byrne Adria de Gispert 22 7 0 16 May 2023
Dual-Alignment Pre-training for Cross-lingual Sentence Embedding Ziheng Li Shaohan Huang Zi-qiang Zhang Zhi-Hong Deng Qiang Lou Haizhen Huang Jian Jiao Furu Wei Weiwei Deng Qi Zhang 28 10 0 16 May 2023
It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performance Arjun Subramonian Xingdi Yuan Hal Daumé Su Lin Blodgett 32 17 0 15 May 2023
Measuring Cross-Lingual Transferability of Multilingual Transformers on Sentence Classification Zewen Chi Heyan Huang Xian-Ling Mao 48 0 0 15 May 2023
Taxi1500: A Multilingual Dataset for Text Classification in 1500 Languages Chunlan Ma Ayyoob Imani Haotian Ye Renhao Pei Ehsaneddin Asgari Hinrich Schütze 22 23 0 15 May 2023
What's the Meaning of Superhuman Performance in Today's NLU? Simone Tedeschi Johan Bos T. Declerck Jan Hajic Daniel Hershcovich ... Simon Krek Steven Schockaert Rico Sennrich Ekaterina Shutova Roberto Navigli ELM LM&MA VLM ReLM LRM 24 26 0 15 May 2023
A Comprehensive Analysis of Adapter Efficiency Nandini Mundra Sumanth Doddapaneni Raj Dabre Anoop Kunchukuttan Ratish Puduppully Mitesh M. Khapra 18 10 0 12 May 2023
A General-Purpose Multilingual Document Encoder Onur Galoglu Robert Litschko Goran Glavas 24 2 0 11 May 2023
GeoGLUE: A GeoGraphic Language Understanding Evaluation Benchmark Dongyang Li Ruixue Ding Qiang-Wei Zhang Zheng Li Boli Chen ... Yao Xu Xin Li Ning Guo Fei Huang Xiaofeng He ELM VLM 23 5 0 11 May 2023
MultiTACRED: A Multilingual Version of the TAC Relation Extraction Dataset Leonhard Hennig Philippe E. Thomas Sebastian Möller 11 8 0 08 May 2023
Test-Time Adaptation with Perturbation Consistency Learning Yi Su Yixin Ji Juntao Li Hai Ye M. Zhang VLM 30 2 0 25 Apr 2023
Domain Mastery Benchmark: An Ever-Updating Benchmark for Evaluating Holistic Domain Knowledge of Large Language Model--A Preliminary Release Zhouhong Gu Xiaoxuan Zhu Haoning Ye Lin Zhang Zhuozhi Xiong Zihan Li Qi He Sihang Jiang Hongwei Feng Yanghua Xiao ELM ALM 42 2 0 23 Apr 2023
UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretraining Hyung Won Chung Noah Constant Xavier Garcia Adam Roberts Yi Tay Sharan Narang Orhan Firat 21 49 0 18 Apr 2023
Romanization-based Large-scale Adaptation of Multilingual Language Models Sukannya Purkayastha Sebastian Ruder Jonas Pfeiffer Iryna Gurevych Ivan Vulić 23 13 0 18 Apr 2023
Transfer to a Low-Resource Language via Close Relatives: The Case Study on Faroese Vésteinn Snaebjarnarson A. Simonsen Goran Glavavs Ivan Vulić 27 19 0 18 Apr 2023
VECO 2.0: Cross-lingual Language Model Pre-training with Multi-granularity Contrastive Learning Zhen-Ru Zhang Chuanqi Tan Songfang Huang Fei Huang VLM 22 5 0 17 Apr 2023
ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning Viet Dac Lai Nghia Trung Ngo Amir Pouran Ben Veyseh Hieu Man Franck Dernoncourt Trung Bui Thien Huu Nguyen ELM LM&MA 25 267 0 12 Apr 2023
Efficiently Aligned Cross-Lingual Transfer Learning for Conversational Tasks using Prompt-Tuning Lifu Tu Jin Qu Semih Yavuz Shafiq R. Joty Wenhao Liu Caiming Xiong Yingbo Zhou 21 7 0 03 Apr 2023
MEGA: Multilingual Evaluation of Generative AI Kabir Ahuja Harshita Diddee Rishav Hada Millicent Ochieng Krithika Ramesh ... T. Ganu Sameer Segal Maxamed Axmed Kalika Bali Sunayana Sitaram LM&MA LRM ELM 19 264 0 22 Mar 2023
Character, Word, or Both? Revisiting the Segmentation Granularity for Chinese Pre-trained Language Models Xinnian Liang Zefan Zhou Hui Huang Shuangzhi Wu Tong Xiao Muyun Yang Zhoujun Li Chao Bian VLM 22 2 0 20 Mar 2023