Large Language Models Only Pass Primary School Exams in Indonesia: A
Comprehensive Test on IndoMMLU

Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU

7 October 2023

Timothy Baldwin

Papers citing "Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU"

10 / 10 papers shown

Title
MultiLoKo: a multilingual local knowledge benchmark for LLMs spanning 31 languages Dieuwke Hupkes Nikolay Bogoychev 46 0 0 14 Apr 2025
LAG-MMLU: Benchmarking Frontier LLM Understanding in Latvian and Giriama Naome A. Etori Kevin Lu Randu Karisa Arturs Kanepajs LRM ELM 46 0 0 14 Mar 2025
MERaLiON-TextLLM: Cross-Lingual Understanding of Large Language Models in Chinese, Indonesian, Malay, and Singlish Xin Huang Tarun K. Vangani Minh Duc Pham Xunlong Zou Bin Wang Zhengyuan Liu A. Aw LRM 34 0 0 21 Dec 2024
Cracking the Code: Multi-domain LLM Evaluation on Real-World Professional Exams in Indonesia Fajri Koto ELM 33 2 0 13 Sep 2024
SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages Holy Lovenia Rahmad Mahendra Salsabil Maulana Akbar Lester James Validad Miranda Jennifer Santoso ... Genta Indra Winata Ruochen Zhang Fajri Koto Zheng-Xin Yong Samuel Cahyawijaya 72 9 0 14 Jun 2024
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 297 3,163 0 21 Mar 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
IndoBERTweet: A Pretrained Language Model for Indonesian Twitter with Effective Domain-Specific Vocabulary Initialization Fajri Koto Jey Han Lau Timothy Baldwin VLM 52 82 0 10 Sep 2021
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics Sebastian Gehrmann Tosin P. Adewumi Karmanya Aggarwal Pawan Sasanka Ammanamanchi Aremu Anuoluwapo ... Nishant Subramani Wei-ping Xu Diyi Yang Akhila Yerukola Jiawei Zhou VLM 238 254 0 02 Feb 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018