Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
2104.07412
Cited By
v1
v2 (latest)
XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021
15 April 2021
Sebastian Ruder
Noah Constant
Jan A. Botha
Aditya Siddhant
Orhan Firat
Jinlan Fu
Pengfei Liu
Junjie Hu
Dan Garrette
Graham Neubig
Melvin Johnson
ELM
AAML
LRM
Re-assign community
ArXiv (abs)
PDF
HTML
Github (644★)
Papers citing
"XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation"
50 / 147 papers shown
Rethinking what Matters: Effective and Robust Multilingual Realignment for Low-Resource Languages
Quang Phuoc Nguyen
David Anugraha
Felix Gaschi
Jun Bin Cheng
En-Shiun Annie Lee
218
0
0
09 Nov 2025
TransAlign: Machine Translation Encoders are Strong Word Aligners, Too
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2025
Benedikt Ebing
Christian Goldschmied
Goran Glavaš
159
0
0
31 Oct 2025
Modality Matching Matters: Calibrating Language Distances for Cross-Lingual Transfer in URIEL+
York Hay Ng
Aditya Khan
Xiang Lu
Matteo Salloum
Michael Zhou
Phuong H. Hoang
A. Seza Doğruöz
En-Shiun Annie Lee
198
1
0
22 Oct 2025
Model-Based Ranking of Source Languages for Zero-Shot Cross-Lingual Transfer
Abteen Ebrahimi
Adam Wiemerslage
Katharina von der Wense
LRM
222
0
0
03 Oct 2025
MENLO: From Preferences to Proficiency -- Evaluating and Modeling Native-like Quality Across 47 Languages
Chenxi Whitehouse
Sebastian Ruder
Tony Lin
Oksana Kurylo
Haruka Takagi
Janice Lam
Nicolò Busetto
Denise Diaz
Francisco Guzmán
194
1
0
30 Sep 2025
Evaluating Language Translation Models by Playing Telephone
Syeda Jannatus Saba
Steven Skiena
148
0
0
23 Sep 2025
SinhalaMMLU: A Comprehensive Benchmark for Evaluating Multitask Language Understanding in Sinhala
Ashmari Pramodya
Nirasha Nelki
Heshan Shalinda
Chamila Liyanage
Yusuke Sakai
Randil Pushpananda
Ruvan Weerasinghe
Hidetaka Kamigaito
Taro Watanabe
LRM
282
1
0
03 Sep 2025
Quantifying Language Disparities in Multilingual Large Language Models
Songbo Hu
Ivan Vulić
Anna Korhonen
150
4
0
23 Aug 2025
Cetvel: A Unified Benchmark for Evaluating Language Understanding, Generation and Cultural Capacity of LLMs for Turkish
Yakup Abrek Er
.Ilker Kesen
Gözde Gül Şahin
Aykut Erdem
ELM
VLM
227
4
0
22 Aug 2025
Survey of NLU Benchmarks Diagnosing Linguistic Phenomena: Why not Standardize Diagnostics Benchmarks?
Khloud Al Jallad
Nada Ghneim
Ghaida Rebdawi
LM&MA
ELM
287
0
0
27 Jul 2025
IndicRAGSuite: Large-Scale Datasets and a Benchmark for Indian Language RAG Systems
Pasunuti Prasanjith
Prathmesh B More
Anoop Kunchukuttan
Mary Dabre
RALM
319
1
0
02 Jun 2025
Moderating Harm: Benchmarking Large Language Models for Cyberbullying Detection in YouTube Comments
International Journal of Computer Applications (IJCA), 2025
Amel Muminovic
ELM
AI4MH
282
0
0
25 May 2025
The Devil Is in the Word Alignment Details: On Translation-Based Cross-Lingual Transfer for Token Classification Tasks
Annual Meeting of the Association for Computational Linguistics (ACL), 2025
Benedikt Ebing
Goran Glavaš
408
2
0
15 May 2025
Myanmar XNLI: Building a Dataset and Exploring Low-resource Approaches to Natural Language Inference with Myanmar
Language Resources and Evaluation (LRE), 2025
Aung Kyaw Htet
Mark Dras
216
7
0
13 Apr 2025
LAG-MMLU: Benchmarking Frontier LLM Understanding in Latvian and Giriama
Naome A. Etori
Kevin Lu
Randu Karisa
Arturs Kanepajs
LRM
ELM
1.0K
4
0
14 Mar 2025
MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation
Weihao Xuan
Rui Yang
Heli Qi
Qingcheng Zeng
Yunze Xiao
...
Edison Marrese-Taylor
Shijian Lu
Yusuke Iwasawa
Yutaka Matsuo
Irene Li
ELM
608
50
0
13 Mar 2025
NusaAksara: A Multimodal and Multilingual Benchmark for Preserving Indonesian Indigenous Scripts
Annual Meeting of the Association for Computational Linguistics (ACL), 2025
Muhammad Farid Adilazuarda
M. Wijanarko
Lucky Susanto
Khumaisa Nuráini
Derry Wijaya
Alham Fikri Aji
408
4
0
25 Feb 2025
URIEL+: Enhancing Linguistic Inclusion and Usability in a Typological and Multilingual Knowledge Base
International Conference on Computational Linguistics (COLING), 2024
Aditya Khan
Mason Shipton
David Anugraha
Kaiyao Duan
Phuong H. Hoang
Eric Khiu
A. Seza Doğruöz
En-Shiun Annie Lee
VLM
430
15
0
17 Feb 2025
INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge
International Conference on Learning Representations (ICLR), 2024
Angelika Romanou
Negar Foroutan
Anna Sotnikova
Zeming Chen
Sree Harsha Nelaturu
...
Mike Zhang
Imanol Schlag
Marzieh Fadaee
Sara Hooker
Antoine Bosselut
ELM
519
44
0
29 Nov 2024
DiffSLT: Enhancing Diversity in Sign Language Translation via Diffusion Model
Pattern Recognition Letters (PR), 2024
JiHwan Moon
Jihoon Park
Jungeun Kim
Jongseong Bae
Hyeongwoo Jeon
Ha Young Kim
326
2
0
26 Nov 2024
Cross-lingual Back-Parsing: Utterance Synthesis from Meaning Representation for Zero-Resource Semantic Parsing
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024
Deokhyung Kang
Seonjeong Hwang
Yunsu Kim
Gary Geunbae Lee
315
0
0
01 Oct 2024
XTRUST: On the Multilingual Trustworthiness of Large Language Models
Yahan Li
Yi Wang
Yi-Ju Chang
Yuan Wu
LRM
HILM
320
2
0
24 Sep 2024
Bilingual Evaluation of Language Models on General Knowledge in University Entrance Exams with Minimal Contamination
International Conference on Computational Linguistics (COLING), 2024
Eva Sánchez Salido
Roser Morante
Julio Gonzalo
Guillermo Marco
Jorge Carrillo-de-Albornoz
...
Enrique Amigó
Andrés Fernández
Alejandro Benito-Santos
Adrián Ghajari Espinosa
Victor Fresno
ELM
334
2
0
19 Sep 2024
AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMs
Basel Mousi
Nadir Durrani
Fatema Ahmad
Md. Arid Hasan
Maram Hasanain
Tameem Kabbani
Fahim Dalvi
Shammur A. Chowdhury
Firoj Alam
423
52
0
17 Sep 2024
Do Large Language Models Speak All Languages Equally? A Comparative Study in Low-Resource Settings
Md. Arid Hasan
Prerona Tarannum
Krishno Dey
Imran Razzak
Usman Naseem
272
10
0
05 Aug 2024
mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024
Xin Zhang
Yanzhao Zhang
Dingkun Long
Wen Xie
Ziqi Dai
...
Pengjun Xie
Fei Huang
Meishan Zhang
Wenjie Li
Min Zhang
357
277
0
29 Jul 2024
sPhinX: Sample Efficient Multilingual Instruction Fine-Tuning Through N-shot Guided Prompting
Sanchit Ahuja
Kumar Tanmay
Hardik Hansrajbhai Chauhan
Barun Patra
Kriti Aggarwal
...
Tejas I. Dhamecha
Ahmed Awadallah
Monojit Choudhary
Vishrav Chaudhary
Sunayana Sitaram
530
4
0
13 Jul 2024
Faux Polyglot: A Study on Information Disparity in Multilingual Large Language Models
Nikhil Sharma
Kenton Murray
Ziang Xiao
551
5
0
07 Jul 2024
Disce aut Deficere: Evaluating LLMs Proficiency on the INVALSI Italian Benchmark
Fabio Mercorio
Mario Mezzanzanica
Daniele Potertì
Antonio Serino
Andrea Seveso
319
10
0
25 Jun 2024
PARIKSHA : A Large-Scale Investigation of Human-LLM Evaluator Agreement on Multilingual and Multi-Cultural Data
Ishaan Watts
Varun Gumma
Aditya Yadavalli
Vivek Seshadri
Manohar Swaminathan
Sunayana Sitaram
ELM
325
29
0
21 Jun 2024
On the Evaluation Practices in Multilingual NLP: Can Machine Translation Offer an Alternative to Human Translations?
Rochelle Choenni
Sara Rajaee
Christof Monz
Ekaterina Shutova
458
6
0
20 Jun 2024
Self-Distillation for Model Stacking Unlocks Cross-Lingual NLU in 200+ Languages
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024
Fabian David Schmidt
Philipp Borchert
Ivan Vulić
Goran Glavaš
291
11
0
18 Jun 2024
Decoding the Diversity: A Review of the Indic AI Research Landscape
Sankalp KJ
Vinija Jain
S. Bhaduri
Tamoghna Roy
Vasu Sharma
328
11
0
13 Jun 2024
MINERS: Multilingual Language Models as Semantic Retrievers
Genta Indra Winata
Ruochen Zhang
David Ifeoluwa Adelani
RALM
491
13
0
11 Jun 2024
From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency
Xenia Ohmer
Elia Bruni
Dieuwke Hupkes
AI4CE
336
11
0
18 Apr 2024
Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers
Libo Qin
Qiguang Chen
Yuhang Zhou
Zhi Chen
Hai-Tao Zheng
Lizi Liao
Min Li
Wanxiang Che
Philip S. Yu
LRM
401
60
0
07 Apr 2024
DIALECTBENCH: A NLP Benchmark for Dialects, Varieties, and Closely-Related Languages
Fahim Faisal
Orevaoghene Ahia
Aarohi Srivastava
Kabir Ahuja
David Chiang
Yulia Tsvetkov
Antonios Anastasopoulos
259
56
0
16 Mar 2024
Cost-Performance Optimization for Processing Low-Resource Language Tasks Using Commercial LLMs
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024
Arijit Nag
Animesh Mukherjee
Niloy Ganguly
Soumen Chakrabarti
294
10
0
08 Mar 2024
Evaluating the Elementary Multilingual Capabilities of Large Language Models with MultiQ
Carolin Holtermann
Paul Röttger
Timm Dill
Anne Lauscher
ELM
LRM
331
36
0
06 Mar 2024
Could We Have Had Better Multilingual LLMs If English Was Not the Central Language?
Ryandito Diandaru
Lucky Susanto
Zilu Tang
Ayu Purwarianti
Derry Wijaya
445
5
0
21 Feb 2024
ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic
Fajri Koto
Jinyan Su
Sara Shatnawi
Jad Doughman
Abdelrahman Boda Sadallah
...
Neha Sengupta
Shady Shehata
Farah E. Shamout
Preslav Nakov
Timothy Baldwin
ELM
LRM
351
85
0
20 Feb 2024
Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning
Annual Meeting of the Association for Computational Linguistics (ACL), 2024
Shivalika Singh
Freddie Vargus
Daniel D'souza
Börje F. Karlsson
Abinaya Mahendiran
...
Max Bartolo
Julia Kreutzer
Ahmet Üstün
Marzieh Fadaee
Sara Hooker
433
192
0
09 Feb 2024
What is "Typological Diversity" in NLP?
Esther Ploeger
Wessel Poelman
Miryam de Lhoneux
Johannes Bjerva
552
5
0
06 Feb 2024
Analyzing the Evaluation of Cross-Lingual Knowledge Transfer in Multilingual Language Models
Sara Rajaee
Christof Monz
293
12
0
03 Feb 2024
Translation Errors Significantly Impact Low-Resource Languages in Cross-Lingual Learning
Ashish Agrawal
Barah Fazili
Preethi Jyothi
278
9
0
03 Feb 2024
ToPro: Token-Level Prompt Decomposition for Cross-Lingual Sequence Labeling Tasks
Bolei Ma
Ercong Nie
Shuzhou Yuan
Helmut Schmid
Michael Farber
Frauke Kreuter
Hinrich Schütze
VLM
374
9
0
29 Jan 2024
Discovering Low-rank Subspaces for Language-agnostic Multilingual Representations
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024
Zhihui Xie
Handong Zhao
Tong Yu
Shuai Li
287
22
0
11 Jan 2024
Understanding LLMs: A Comprehensive Overview from Training to Inference
Yi-Hsueh Liu
Haoyang He
Tianle Han
Xu-Yao Zhang
Mengyuan Liu
...
Xiaoyan Cai
Tuo Zhang
Ning Qiang
Tianming Liu
Bao Ge
SyDa
502
139
0
04 Jan 2024
To Translate or Not to Translate: A Systematic Investigation of Translation-Based Cross-Lingual Transfer to Low-Resource Languages
North American Chapter of the Association for Computational Linguistics (NAACL), 2023
Benedikt Ebing
Goran Glavaš
288
10
0
15 Nov 2023
PLUG: Leveraging Pivot Language in Cross-Lingual Instruction Tuning
Annual Meeting of the Association for Computational Linguistics (ACL), 2023
Zhihan Zhang
Dong-Ho Lee
Yuwei Fang
Wenhao Yu
Mengzhao Jia
Meng Jiang
Francesco Barbieri
ALM
451
44
0
15 Nov 2023
1
2
3
Next
Page 1 of 3
