XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization

24 March 2020

Graham Neubig

Papers citing "XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization"

50 / 659 papers shown

Title
A Systematic Study of Performance Disparities in Multilingual Task-Oriented Dialogue Systems Songbo Hu Han Zhou Moy Yuan Milan Gritta Guchun Zhang Ignacio Iacobacci Anna Korhonen Ivan Vulić 26 3 0 19 Oct 2023
One For All & All For One: Bypassing Hyperparameter Tuning with Model Averaging For Cross-Lingual Transfer Fabian David Schmidt Ivan Vulić Goran Glavavs MoMe 11 3 0 16 Oct 2023
Crosslingual Structural Priming and the Pre-Training Dynamics of Bilingual Language Models Catherine Arnett Tyler A. Chang J. Michaelov Benjamin Bergen 11 0 0 11 Oct 2023
Unleashing the Multilingual Encoder Potential: Boosting Zero-Shot Performance via Probability Calibration Ercong Nie Helmut Schmid Hinrich Schütze UQCV 16 2 0 08 Oct 2023
Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU Fajri Koto Nurul Aisyah Haonan Li Timothy Baldwin AI4Ed LRM ELM 30 37 0 07 Oct 2023
A Brief History of Prompt: Leveraging Language Models. (Through Advanced Prompting) G. Muktadir SILM 29 8 0 30 Sep 2023
Self-Augmentation Improves Zero-Shot Cross-Lingual Transfer Fei Wang Kuan-Hao Huang Kai-Wei Chang Muhao Chen 18 4 0 19 Sep 2023
NusaWrites: Constructing High-Quality Corpora for Underrepresented and Extremely Low-Resource Languages Samuel Cahyawijaya Holy Lovenia Fajri Koto Dea Adhista Emmanuel Dave ... Genta Indra Winata David Moeljadi Alham Fikri Aji Ayu Purwarianti Pascale Fung 41 7 0 19 Sep 2023
OpenMSD: Towards Multilingual Scientific Documents Similarity Measurement Yang Gao Ji Ma I. Korotkov Keith B. Hall Dana Alon Donald Metzler 11 0 0 19 Sep 2023
Contextual Label Projection for Cross-Lingual Structured Prediction Tanmay Parekh I-Hung Hsu Kuan-Hao Huang Kai-Wei Chang Nanyun Peng 24 4 0 16 Sep 2023
Leveraging Multi-lingual Positive Instances in Contrastive Learning to Improve Sentence Embedding Kaiyan Zhao Qiyu Wu Xin-Qiang Cai Yoshimasa Tsuruoka 25 6 0 16 Sep 2023
SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects David Ifeoluwa Adelani Hannah Liu Xiaoyu Shen Nikita Vassilyev Jesujoba Oluwadara Alabi Yanke Mao Haonan Gao Annie En-Shiun Lee ELM 27 59 0 14 Sep 2023
BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Models Wei Qi Leong Jian Gang Ngui Yosephine Susanto Hamsawardhini Rengarajan Kengatharaiyer Sarveswaran William-Chandra Tjhi 21 9 0 12 Sep 2023
MADLAD-400: A Multilingual And Document-Level Large Audited Dataset Sneha Kudugunta Isaac Caswell Biao Zhang Xavier Garcia Christopher A. Choquette-Choo ... Derrick Xin Aditya Kusupati Romi Stella Ankur Bapna Orhan Firat 59 118 0 09 Sep 2023
Multiple Representation Transfer from Large Language Models to End-to-End ASR Systems Takuma Udagawa Masayuki Suzuki Gakuto Kurata Masayasu Muraoka G. Saon 30 2 0 07 Sep 2023
Multilingual Text Representation Fahim Faisal 19 0 0 02 Sep 2023
mCL-NER: Cross-Lingual Named Entity Recognition via Multi-view Contrastive Learning Ying Mo Jian Yang Jiahao Liu Qifan Wang Ruoyu Chen Jingang Wang Zhoujun Li 13 23 0 17 Aug 2023
Factuality Detection using Machine Translation -- a Use Case for German Clinical Text Mohammed Mustafa Ahmed Bin Sumait Aleksandra Gabryszak Leonhard Hennig Roland Roller MedIm HILM 14 0 0 17 Aug 2023
A User-Centered Evaluation of Spanish Text Simplification Adrian de Wynter Anthony Hevia Si-Qing Chen 21 0 0 15 Aug 2023
Cross-Lingual Constituency Parsing for Middle High German: A Delexicalized Approach Ercong Nie Helmut Schmid Hinrich Schütze 16 1 0 09 Aug 2023
Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback Viet Dac Lai Chien Van Nguyen Nghia Trung Ngo Thuat Nguyen Franck Dernoncourt Ryan A. Rossi Thien Huu Nguyen ALM 33 127 0 29 Jul 2023
Multi3WOZ: A Multilingual, Multi-Domain, Multi-Parallel Dataset for Training and Evaluating Culturally Adapted Task-Oriented Dialog Systems Songbo Hu Han Zhou Mete Hergul Milan Gritta Guchun Zhang Ignacio Iacobacci Ivan Vulić Anna Korhonen 24 10 0 26 Jul 2023
A Dataset and Strong Baselines for Classification of Czech News Texts Hynek Kydlívcek Jindrich Libovický 11 0 0 20 Jul 2023
Gradient Sparsification For Masked Fine-Tuning of Transformers J. Ó. Neill Sourav Dutta 14 0 0 19 Jul 2023
Is Prompt-Based Finetuning Always Better than Vanilla Finetuning? Insights from Cross-Lingual Language Understanding Bolei Ma Ercong Nie Helmut Schmid Hinrich Schütze AAML VLM LRM 26 8 0 15 Jul 2023
Empowering Cross-lingual Behavioral Testing of NLP Models with Typological Features Ester Hlavnova Sebastian Ruder 30 5 0 11 Jul 2023
Enhancing Cross-lingual Transfer via Phonemic Transcription Integration Hoang Nguyen Chenwei Zhang Tao Zhang Eugene Rohrbaugh Philip S. Yu 10 7 0 10 Jul 2023
Optimal Transport Posterior Alignment for Cross-lingual Semantic Parsing Tom Sherborne Tom Hosking Mirella Lapata OT 24 4 0 09 Jul 2023
Stop Pre-Training: Adapt Visual-Language Models to Unseen Languages Yasmine Karoui R. Lebret Negar Foroutan Karl Aberer MLLM VLM 24 1 0 29 Jun 2023
SkillNet-X: A Multilingual Multitask Model with Sparsely Activated Skills Zhangyin Feng Yong Dai Fan Zhang Duyu Tang Xiaocheng Feng Shuangzhi Wu Bing Qin Yunbo Cao Shuming Shi MoE 21 0 0 28 Jun 2023
Cross-Lingual Cross-Age Group Adaptation for Low-Resource Elderly Speech Emotion Recognition Samuel Cahyawijaya Holy Lovenia Willy Chung Rita Frieske Zihan Liu Pascale Fung 37 1 0 26 Jun 2023
L3Cube-MahaSent-MD: A Multi-domain Marathi Sentiment Analysis Dataset and Transformer Models Aabha Pingle Aditya Vyawahare Isha Joshi Rahul Tangsali Raviraj Joshi 12 8 0 24 Jun 2023
Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the US J. H. Rystrøm 11 0 0 22 Jun 2023
Recipes for Sequential Pre-training of Multilingual Encoder and Seq2Seq Models Saleh Soltan Andrew Rosenbaum Tobias Falke Qin Lu Anna Rumshisky Wael Hamza 14 0 0 14 Jun 2023
Babel-ImageNet: Massively Multilingual Evaluation of Vision-and-Language Representations Gregor Geigle Radu Timofte Goran Glavas VLM MLLM 21 5 0 14 Jun 2023
Massively Multilingual Corpus of Sentiment Datasets and Multi-faceted Sentiment Classification Benchmark Lukasz Augustyniak Szymon Wo'zniak Marcin Gruza Piotr Gramacki Krzysztof Rajda M. Morzy Tomasz Kajdanowicz 20 5 0 13 Jun 2023
Soft Language Clustering for Multilingual Model Pre-training Jiali Zeng Yu Jiang Yongjing Yin Yingqi Jing Fandong Meng Binghuai Lin Yunbo Cao Jie Zhou 17 4 0 13 Jun 2023
Multi-Source Test-Time Adaptation as Dueling Bandits for Extractive Question Answering Hai Ye Qizhe Xie Hwee Tou Ng 32 8 0 11 Jun 2023
Language Versatilists vs. Specialists: An Empirical Revisiting on Multilingual Transfer Ability Jiacheng Ye Xijia Tao Lingpeng Kong LRM 28 22 0 11 Jun 2023
M3Exam: A Multilingual, Multimodal, Multilevel Benchmark for Examining Large Language Models Wenxuan Zhang Sharifah Mahani Aljunied Chang Gao Yew Ken Chia Lidong Bing ELM 21 81 0 08 Jun 2023
T3L: Translate-and-Test Transfer Learning for Cross-Lingual Text Classification Inigo Jauregi Unanue Gholamreza Haffari Massimo Piccardi VLM 24 8 0 08 Jun 2023
Can current NLI systems handle German word order? Investigating language model performance on a new German challenge set of minimal pairs Ines Reinig K. Markert 8 0 0 07 Jun 2023
Multilingual Clinical NER: Translation or Cross-lingual Transfer? X. Fontaine Félix Gaschi Parisa Rastin Y. Toussaint 24 8 0 07 Jun 2023
XSemPLR: Cross-Lingual Semantic Parsing in Multiple Natural Languages and Meaning Representations Yusen Zhang J. Wang Zhiguo Wang Rui Zhang VLM 57 9 0 07 Jun 2023
Cross-Lingual Transfer with Target Language-Ready Task Adapters Marinela Parović Alan Ansell Ivan Vulić Anna Korhonen 16 8 0 05 Jun 2023
bgGLUE: A Bulgarian General Language Understanding Evaluation Benchmark Momchil Hardalov Pepa Atanasova Todor Mihaylov G. Angelova K. Simov P. Osenova Ves Stoyanov Ivan Koychev Preslav Nakov Dragomir R. Radev ELM FedML 21 4 0 04 Jun 2023
Exploring Anisotropy and Outliers in Multilingual Language Models for Cross-Lingual Semantic Sentence Similarity Katharina Hämmerl Alina Fastowski Jindrich Libovický Alexander M. Fraser 20 6 0 01 Jun 2023
What does the Failure to Reason with "Respectively" in Zero/Few-Shot Settings Tell Us about Language Models? Ruixiang Cui Seolhwa Lee Daniel Hershcovich Anders Søgaard 25 2 0 31 May 2023
Translation-Enhanced Multilingual Text-to-Image Generation Yaoyiran Li Ching-Yun Chang Stephen Rawls Ivan Vulić Anna Korhonen 14 8 0 30 May 2023
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets Md Tahmid Rahman Laskar M Saiful Bari Mizanur Rahman Md Amran Hossen Bhuiyan Shafiq R. Joty J. Huang LM&MA ELM ALM 41 178 0 29 May 2023