MiLiC-Eval: Benchmarking Multilingual LLMs for China's Minority Languages

Large language models (LLMs) excel in high-resource languages but struggle with low-resource languages (LRLs), particularly those spoken by minority communities in China, such as Tibetan, Uyghur, Kazakh, and Mongolian. To systematically track the progress in these languages, we introduce MiLiC-Eval, a benchmark designed for minority languages in China, featuring 24K instances across 9 tasks. MiLiC-Eval focuses on underrepresented writing systems and provides a fine-grained assessment of linguistic and problem-solving skills. Our evaluation reveals that LLMs perform poorly on syntax-intensive tasks and multi-script languages. We further demonstrate how MiLiC-Eval can help advance LRL research in handling diverse writing systems and understanding the process of language adaptation.

@article{zhang2025_2503.01150,
  title={ MiLiC-Eval: Benchmarking Multilingual LLMs for China's Minority Languages },
  author={ Chen Zhang and Mingxu Tao and Zhiyuan Liao and Yansong Feng },
  journal={arXiv preprint arXiv:2503.01150},
  year={ 2025 }
}