XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization

24 March 2020

Graham Neubig

Papers citing "XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization"

50 / 659 papers shown

Title
Assessing the Role of Lexical Semantics in Cross-lingual Transfer through Controlled Manipulations Roy Ilani Taelin Karidi Omri Abend 19 0 0 14 Aug 2024
Do Large Language Models Speak All Languages Equally? A Comparative Study in Low-Resource Settings Md. Arid Hasan Prerona Tarannum Krishno Dey Imran Razzak Usman Naseem 29 4 0 05 Aug 2024
mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval Xin Zhang Yanzhao Zhang Dingkun Long Wen Xie Ziqi Dai ... Pengjun Xie Fei Huang Meishan Zhang Wenjie Li Min Zhang 35 73 0 29 Jul 2024
Multilingual Fine-Grained News Headline Hallucination Detection Jiaming Shen Tianqi Liu Jialu Liu Zhen Qin Jay Pavagadhi Simon Baumgartner Michael Bendersky 39 0 0 22 Jul 2024
MASIVE: Open-Ended Affective State Identification in English and Spanish Nicholas Deas Elsbeth Turcan Iván Pérez Mejía Kathleen McKeown CVBM 19 0 0 16 Jul 2024
sPhinX: Sample Efficient Multilingual Instruction Fine-Tuning Through N-shot Guided Prompting Sanchit Ahuja Kumar Tanmay Hardik Hansrajbhai Chauhan Barun Patra Kriti Aggarwal ... Tejas I. Dhamecha Ahmed Awadallah Monojit Choudhary Vishrav Chaudhary Sunayana Sitaram 27 3 0 13 Jul 2024
NativQA: Multilingual Culturally-Aligned Natural Query for LLMs Md. Arid Hasan Maram Hasanain Fatema Ahmad Sahinur Rahman Laskar Sunaya Upadhyay Vrunda N. Sukhadia Mucahid Kutlu Shammur A. Chowdhury Firoj Alam 45 4 0 13 Jul 2024
MAGNET: Improving the Multilingual Fairness of Language Models with Adaptive Gradient-Based Tokenization Orevaoghene Ahia Sachin Kumar Hila Gonen Valentin Hoffman Tomasz Limisiewicz Yulia Tsvetkov Noah A. Smith 43 4 0 11 Jul 2024
Data, Data Everywhere: A Guide for Pretraining Dataset Construction Jupinder Parmar Shrimai Prabhumoye Joseph Jennings Bo Liu Aastha Jhunjhunwala Zhilin Wang M. Patwary M. Shoeybi Bryan Catanzaro 34 5 0 08 Jul 2024
IL-TUR: Benchmark for Indian Legal Text Understanding and Reasoning Abhinav Joshi Shounak Paul Akshat Sharma Pawan Goyal Saptarshi Ghosh Ashutosh Modi AILaw ELM 31 7 0 07 Jul 2024
Faux Polyglot: A Study on Information Disparity in Multilingual Large Language Models Nikhil Sharma Kenton Murray Ziang Xiao 50 1 0 07 Jul 2024
Cross-Lingual Word Alignment for ASEAN Languages with Contrastive Learning Jingshen Zhang Xinying Qiu Teng Shen Wenyu Wang Kailin Zhang Wenhe Feng 35 0 0 06 Jul 2024
Soft Language Prompts for Language Transfer Ivan Vykopal Simon Ostermann Marián Simko AAML 37 1 0 02 Jul 2024
GemmAr: Enhancing LLMs Through Arabic Instruction-Tuning Hasna Chouikhi Manel Aloui Cyrine Ben Hammou Ghaith Chaabane Haithem Kchaou Chehir Dhaouadi 26 0 0 02 Jul 2024
M2QA: Multi-domain Multilingual Question Answering Leon Arne Engländer Hannah Sterz Clifton A. Poth Jonas Pfeiffer Ilia Kuznetsov Iryna Gurevych VLM 33 1 0 01 Jul 2024
Self-Translate-Train: A Simple but Strong Baseline for Cross-lingual Transfer of Large Language Models Ryokan Ri Shun Kiyono Sho Takase SyDa 21 0 0 29 Jun 2024
Understanding and Mitigating Language Confusion in LLMs Kelly Marchisio Wei-Yin Ko Alexandre Berard Théo Dehaze Sebastian Ruder 56 23 0 28 Jun 2024
Teaching LLMs to Abstain across Languages via Multilingual Feedback Shangbin Feng Weijia Shi Yike Wang Wenxuan Ding Orevaoghene Ahia Shuyue Stella Li Vidhisha Balachandran Sunayana Sitaram Yulia Tsvetkov 65 4 0 22 Jun 2024
PARIKSHA : A Large-Scale Investigation of Human-LLM Evaluator Agreement on Multilingual and Multi-Cultural Data Ishaan Watts Varun Gumma Aditya Yadavalli Vivek Seshadri Manohar Swaminathan Sunayana Sitaram ELM 38 8 0 21 Jun 2024
Data-Centric AI in the Age of Large Language Models Xinyi Xu Zhaoxuan Wu Rui Qiao Arun Verma Yao Shu ... Xiaoqiang Lin Wenyang Hu Zhongxiang Dai Pang Wei Koh Bryan Kian Hsiang Low ALM 40 2 0 20 Jun 2024
On the Evaluation Practices in Multilingual NLP: Can Machine Translation Offer an Alternative to Human Translations? Rochelle Choenni Sara Rajaee Christof Monz Ekaterina Shutova 31 1 0 20 Jun 2024
Towards Robust Evaluation: A Comprehensive Taxonomy of Datasets and Metrics for Open Domain Question Answering in the Era of Large Language Models Akchay Srivastava Atif Memon ELM 40 1 0 19 Jun 2024
Probing the Emergence of Cross-lingual Alignment during LLM Training Hetong Wang Pasquale Minervini E. Ponti 25 7 0 19 Jun 2024
Self-Distillation for Model Stacking Unlocks Cross-Lingual NLU in 200+ Languages Fabian David Schmidt Philipp Borchert Ivan Vulić Goran Glavaš 42 5 0 18 Jun 2024
OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI Zhen Huang Zengzhi Wang Shijie Xia Xuefeng Li Haoyang Zou ... Yuxiang Zheng Shaoting Zhang Dahua Lin Yu Qiao Pengfei Liu ELM LRM 43 25 0 18 Jun 2024
UniBridge: A Unified Approach to Cross-Lingual Transfer Learning for Low-Resource Languages Trinh Pham Khoi M. Le Luu Anh Tuan 29 1 0 14 Jun 2024
Decipherment-Aware Multilingual Learning in Jointly Trained Language Models Grandee Lee 29 0 0 11 Jun 2024
Thai Winograd Schemas: A Benchmark for Thai Commonsense Reasoning Phakphum Artkaew LRM 20 0 0 28 May 2024
Exploring Alignment in Shared Cross-lingual Spaces Basel Mousi Nadir Durrani Fahim Dalvi Majd Hawasly Ahmed Abdelali 30 0 0 23 May 2024
XAMPLER: Learning to Retrieve Cross-Lingual In-Context Examples Peiqin Lin André F. T. Martins Hinrich Schütze RALM 45 2 0 08 May 2024
Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities Kazuki Fujii Taishi Nakamura Mengsay Loem Hiroki Iida Masanari Ohi Kakeru Hattori Hirai Shota Sakae Mizuki Rio Yokota Naoaki Okazaki CLL 35 53 0 27 Apr 2024
IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages Harman Singh Nitish Gupta Shikhar Bharadwaj Dinesh Tewari Partha P. Talukdar ELM 32 22 0 25 Apr 2024
Incorporating Lexical and Syntactic Knowledge for Unsupervised Cross-Lingual Transfer Jianyu Zheng Fengfei Fan Jianquan Li 16 2 0 25 Apr 2024
Holistic Safety and Responsibility Evaluations of Advanced AI Models Laura Weidinger Joslyn Barnhart Jenny Brennan Christina Butterfield Susie Young ... Sebastian Farquhar Lewis Ho Iason Gabriel Allan Dafoe William S. Isaac ELM 27 8 0 22 Apr 2024
CORI: CJKV Benchmark with Romanization Integration -- A step towards Cross-lingual Transfer Beyond Textual Scripts Hoang Nguyen Chenwei Zhang Ye Liu Natalie Parde Eugene Rohrbaugh Philip S. Yu 47 1 0 19 Apr 2024
Reuse Your Rewards: Reward Model Transfer for Zero-Shot Cross-Lingual Alignment Zhaofeng Wu Ananth Balashankar Yoon Kim Jacob Eisenstein Ahmad Beirami 46 8 0 18 Apr 2024
From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency Xenia Ohmer Elia Bruni Dieuwke Hupkes AI4CE 31 6 0 18 Apr 2024
GeMQuAD : Generating Multilingual Question Answering Datasets from Large Language Models using Few Shot Learning Amani Namboori Shivam Mangale Andrew Rosenbaum Saleh Soltan 40 0 0 14 Apr 2024
Understanding Cross-Lingual Alignment -- A Survey Katharina Hämmerl Jindvrich Libovický Alexander M. Fraser 36 10 0 09 Apr 2024
PORTULAN ExtraGLUE Datasets and Models: Kick-starting a Benchmark for the Neural Processing of Portuguese T. Osório Bernardo Leite Henrique Lopes Cardoso Luís Gomes João Rodrigues Rodrigo Santos António Branco 27 3 0 08 Apr 2024
Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers Libo Qin Qiguang Chen Yuhang Zhou Zhi Chen Yinghui Li Lizi Liao Min Li Wanxiang Che Philip S. Yu LRM 47 36 0 07 Apr 2024
A Morphology-Based Investigation of Positional Encodings Poulami Ghosh Shikhar Vashishth Raj Dabre Pushpak Bhattacharyya 24 1 0 06 Apr 2024
Enhancing Cross-lingual Sentence Embedding for Low-resource Languages with Word Alignment Zhongtao Miao Qiyu Wu Kaiyan Zhao Zilong Wu Yoshimasa Tsuruoka 28 9 0 03 Apr 2024
AAdaM at SemEval-2024 Task 1: Augmentation and Adaptation for Multilingual Semantic Textual Relatedness Miaoran Zhang Mingyang Wang Jesujoba Oluwadara Alabi Dietrich Klakow VLM 33 4 0 01 Apr 2024
Effectively Prompting Small-sized Language Models for Cross-lingual Tasks via Winning Tickets Mingqi Li Feng Luo LRM 27 0 0 01 Apr 2024
A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias Yuemei Xu Ling Hu Jiayi Zhao Zihan Qiu Yuqi Ye Hanwen Gu LRM 19 36 0 01 Apr 2024
Cross-Lingual Transfer Robustness to Lower-Resource Languages on Adversarial Datasets Shadi Manafi Nikhil Krishnaswamy AAML 35 0 0 29 Mar 2024
WaterJudge: Quality-Detection Trade-off when Watermarking Large Language Models Piotr Molenda Adian Liusie Mark J. F. Gales WaLM 52 4 0 28 Mar 2024
Multilingual Sentence-T5: Scalable Sentence Encoders for Multilingual Applications Chihiro Yano Akihiko Fukuchi Shoko Fukasawa Hideyuki Tachibana Yotaro Watanabe 34 2 0 26 Mar 2024
DIALECTBENCH: A NLP Benchmark for Dialects, Varieties, and Closely-Related Languages Fahim Faisal Orevaoghene Ahia Aarohi Srivastava Kabir Ahuja David Chiang Yulia Tsvetkov Antonios Anastasopoulos 56 27 0 16 Mar 2024