XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization

24 March 2020

Graham Neubig

Papers citing "XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization"

50 / 659 papers shown

Title
CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Korean Eunsu Kim Juyoung Suk Philhoon Oh Haneul Yoo James Thorne Alice H. Oh ELM 67 15 0 11 Mar 2024
Cost-Performance Optimization for Processing Low-Resource Language Tasks Using Commercial LLMs Arijit Nag Animesh Mukherjee Niloy Ganguly Soumen Chakrabarti 33 2 0 08 Mar 2024
NLPre: a revised approach towards language-centric benchmarking of Natural Language Preprocessing systems Martyna Wia̧cek Piotr Rybak Lukasz Pszenny Alina Wróblewska 21 1 0 07 Mar 2024
A Measure for Transparent Comparison of Linguistic Diversity in Multilingual NLP Data Sets Tanja Samardzic Ximena Gutierrez-Vasques Christian Bentz Steven Moran Olga Pelloni 24 4 0 06 Mar 2024
From One to Many: Expanding the Scope of Toxicity Mitigation in Language Models Luiza Amador Pozzobon Patrick Lewis Sara Hooker B. Ermiş 36 7 0 06 Mar 2024
Exploring the Limitations of Large Language Models in Compositional Relation Reasoning Jinman Zhao Xueyan Zhang BDL LRM 25 4 0 05 Mar 2024
Improving Cross-lingual Representation for Semantic Retrieval with Code-switching M. Maimaiti Yuanhang Zheng Ji Zhang Fei Huang Yue Zhang Wenpei Luo Kaiyu Huang LRM 23 0 0 03 Mar 2024
VNLP: Turkish NLP Package Meliksah Turker Mehmet Erdi Ari Aydin Han 37 1 0 02 Mar 2024
TRUCE: Private Benchmarking to Prevent Contamination and Improve Comparative Evaluation of LLMs Tanmay Rajore Nishanth Chandran Sunayana Sitaram Divya Gupta Rahul Sharma Kashish Mittal Manohar Swaminathan 39 14 0 01 Mar 2024
Decomposed Prompting: Unveiling Multilingual Linguistic Structure Knowledge in English-Centric Large Language Models Ercong Nie Shuzhou Yuan Bolei Ma Helmut Schmid Michael Farber Frauke Kreuter Hinrich Schütze ReLM 91 6 0 28 Feb 2024
Multi-Task Contrastive Learning for 8192-Token Bilingual Text Embeddings Isabelle Mohr Markus Krimmel Saba Sturua Mohammad Kalim Akram Andreas Koukounas ... Susana Guzman Bo Wang Maximilian Werk Nan Wang Han Xiao 30 15 0 26 Feb 2024
Mitigating the Linguistic Gap with Phonemic Representations for Robust Multilingual Language Understanding Haeji Jung Changdae Oh Jooeon Kang Jimin Sohn Kyungwoo Song Jinkyu Kim David R. Mortensen 24 0 0 22 Feb 2024
Analysis of Multi-Source Language Training in Cross-Lingual Transfer Seong Hoon Lim Taejun Yun Jinhyeon Kim Jihun Choi Taeuk Kim 46 2 0 21 Feb 2024
ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic Fajri Koto Haonan Li Sara Shatnawi Jad Doughman Abdelrahman Boda Sadallah ... Neha Sengupta Shady Shehata Nizar Habash Preslav Nakov Timothy Baldwin ELM LRM 72 30 0 20 Feb 2024
Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model A. Ustun Viraat Aryabumi Zheng-Xin Yong Wei-Yin Ko Daniel D'souza ... Shayne Longpre Niklas Muennighoff Marzieh Fadaee Julia Kreutzer Sara Hooker ALM ELM SyDa LRM 27 193 0 12 Feb 2024
Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning Shivalika Singh Freddie Vargus Daniel D'souza Börje F. Karlsson Abinaya Mahendiran ... Max Bartolo Julia Kreutzer A. Ustun Marzieh Fadaee Sara Hooker 117 115 0 09 Feb 2024
LESS: Selecting Influential Data for Targeted Instruction Tuning Mengzhou Xia Sadhika Malladi Suchin Gururangan Sanjeev Arora Danqi Chen 80 185 0 06 Feb 2024
Constrained Decoding for Cross-lingual Label Projection Duong Minh Le Yang Chen Alan Ritter Wei-ping Xu 19 7 0 05 Feb 2024
Analyzing the Evaluation of Cross-Lingual Knowledge Transfer in Multilingual Language Models Sara Rajaee Christof Monz 19 3 0 03 Feb 2024
Translation Errors Significantly Impact Low-Resource Languages in Cross-Lingual Learning Ashish Agrawal Barah Fazili P. Jyothi 19 2 0 03 Feb 2024
Exploring the Robustness of Task-oriented Dialogue Systems for Colloquial German Varieties Ekaterina Artemova Verena Blaschke Barbara Plank 15 3 0 03 Feb 2024
Evaluation Methodology for Large Language Models for Multilingual Document Question and Answer Adar Kahana Jaya Susan Mathew Said Bleik Jeremy Reynolds Oren Elisha 19 0 0 01 Feb 2024
Can Large Language Models be Trusted for Evaluation? Scalable Meta-Evaluation of LLMs as Evaluators via Agent Debate Steffi Chern Ethan Chern Graham Neubig Pengfei Liu LLMAG ALM ELM 11 25 0 30 Jan 2024
ToPro: Token-Level Prompt Decomposition for Cross-Lingual Sequence Labeling Tasks Bolei Ma Ercong Nie Shuzhou Yuan Helmut Schmid Michael Farber Frauke Kreuter Hinrich Schütze VLM 95 5 0 29 Jan 2024
TransliCo: A Contrastive Learning Framework to Address the Script Barrier in Multilingual Pretrained Language Models Yihong Liu Chunlan Ma Haotian Ye Hinrich Schütze 23 1 0 12 Jan 2024
Discovering Low-rank Subspaces for Language-agnostic Multilingual Representations Zhihui Xie Handong Zhao Tong Yu Shuai Li 26 13 0 11 Jan 2024
Understanding LLMs: A Comprehensive Overview from Training to Inference Yi-Hsueh Liu Haoyang He Tianle Han Xu-Yao Zhang Mengyuan Liu ... Xintao Hu Tuo Zhang Ning Qiang Tianming Liu Bao Ge SyDa 14 64 0 04 Jan 2024
Multilingual Instruction Tuning With Just a Pinch of Multilinguality Uri Shaham Jonathan Herzig Roee Aharoni Idan Szpektor Reut Tsarfaty Matan Eyal LRM 37 41 0 03 Jan 2024
The Obscure Limitation of Modular Multilingual Language Models Muhammad Farid Adilazuarda Samuel Cahyawijaya Ayu Purwarianti 19 6 0 21 Nov 2023
To Translate or Not to Translate: A Systematic Investigation of Translation-Based Cross-Lingual Transfer to Low-Resource Languages Benedikt Ebing Goran Glavas 21 4 0 15 Nov 2023
When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages Tyler A. Chang Catherine Arnett Zhuowen Tu Benjamin Bergen LRM 20 7 0 15 Nov 2023
MELA: Multilingual Evaluation of Linguistic Acceptability Ziyin Zhang Yikang Liu Wei Huang Junyu Mao Rui Wang Hai Hu 22 3 0 15 Nov 2023
OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretraining Yihong Liu Peiqin Lin Mingyang Wang Hinrich Schütze 19 21 0 15 Nov 2023
A Material Lens on Coloniality in NLP William B. Held Camille Harris Michael Best Diyi Yang 20 11 0 14 Nov 2023
Sharing, Teaching and Aligning: Knowledgeable Transfer Learning for Cross-Lingual Machine Reading Comprehension Tingfeng Cao Chengyu Wang Chuanqi Tan Jun Huang Jinhui Zhu 6 0 0 12 Nov 2023
From Classification to Generation: Insights into Crosslingual Retrieval Augmented ICL Xiaoqian Li Ercong Nie Sheng Liang RALM LRM 100 10 0 11 Nov 2023
Zero-Shot Cross-Lingual Sentiment Classification under Distribution Shift: an Exploratory Study Maarten De Raedt Semere Kiros Bitew Fréderic Godin Thomas Demeester Chris Develder 48 4 0 11 Nov 2023
DeMuX: Data-efficient Multilingual Learning Simran Khanuja Srinivas Gowriraj Lucio Dery Graham Neubig VLM 23 1 0 10 Nov 2023
Bridging the Digital Divide: Performance Variation across Socio-Economic Factors in Vision-Language Models Joan Nwatu Oana Ignat Rada Mihalcea 18 9 0 09 Nov 2023
Self-Influence Guided Data Reweighting for Language Model Pre-training Megh Thakkar Tolga Bolukbasi Sriram Ganapathy Shikhar Vashishth Sarath Chandar Partha P. Talukdar MILM 22 20 0 02 Nov 2023
Crosslingual Retrieval Augmented In-context Learning for Bangla Xiaoqian Li Ercong Nie Sheng Liang RALM LRM 57 7 0 01 Nov 2023
EMMA-X: An EM-like Multilingual Pre-training Algorithm for Cross-lingual Representation Learning Ping Guo Xiangpeng Wei Yue Hu Baosong Yang Dayiheng Liu Fei Huang Jun Xie 21 2 0 26 Oct 2023
X-SNS: Cross-Lingual Transfer Prediction through Sub-Network Similarity Taejun Yun Jinhyeon Kim Deokyeong Kang Seong Hoon Lim Jihoon Kim Taeuk Kim 27 0 0 26 Oct 2023
A Multilingual Virtual Guide for Self-Attachment Technique Alicia Jiayun Law Ruoyu Hu Lisa Alazraki Anandha Gopalan Neophytos Polydorou A. Edalat 11 3 0 25 Oct 2023
XFEVER: Exploring Fact Verification across Languages Yi-Chen Chang Canasai Kruengkrai Junichi Yamagishi HILM 11 3 0 25 Oct 2023
A Joint Matrix Factorization Analysis of Multilingual Representations Zheng Zhao Yftah Ziser Bonnie Webber Shay B. Cohen 16 2 0 24 Oct 2023
GradSim: Gradient-Based Language Grouping for Effective Multilingual Training Mingyang Wang Heike Adel Lukas Lange Jannik Strötgen Hinrich Schütze 20 3 0 23 Oct 2023
Breaking the Language Barrier: Improving Cross-Lingual Reasoning with Structured Self-Attention Negar Foroutan Mohammadreza Banaei Karl Aberer Antoine Bosselut LRM 17 3 0 23 Oct 2023
The Skipped Beat: A Study of Sociopragmatic Understanding in LLMs for 64 Languages Chiyu Zhang Khai Duy Doan Qisheng Liao Muhammad Abdul-Mageed 34 6 0 23 Oct 2023
Semantic and Expressive Variation in Image Captions Across Languages Andre Ye Sebastin Santy Jena D. Hwang Amy X. Zhang Ranjay Krishna VLM 46 3 0 22 Oct 2023