CMMLU: Measuring massive multitask language understanding in Chinese

15 June 2023

Papers citing "CMMLU: Measuring massive multitask language understanding in Chinese"

45 / 45 papers shown

Title
ReplaceMe: Network Simplification via Layer Pruning and Linear Transformations Dmitriy Shopkhoev Ammar Ali Magauiya Zhussip Valentin Malykh Stamatios Lefkimmiatis N. Komodakis Sergey Zagoruyko VLM 78 0 0 05 May 2025
Measuring Hong Kong Massive Multi-Task Language Understanding Chuxue Cao Zhenghao Zhu Junqi Zhu Guoying Lu Siyu Peng Juntao Dai Weijie Shi Sirui Han Yike Guo ELM 82 0 0 04 May 2025
MultiLoKo: a multilingual local knowledge benchmark for LLMs spanning 31 languages Dieuwke Hupkes Nikolay Bogoychev 91 0 0 14 Apr 2025
InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models Jinguo Zhu Weiyun Wang Zhe Chen Z. Liu Shenglong Ye ... D. Lin Yu Qiao Jifeng Dai Wenhai Wang W. Wang MLLM VLM 66 7 1 14 Apr 2025
CARE: Aligning Language Models for Regional Cultural Awareness Geyang Guo Tarek Naous Hiromi Wakaki Yukiko Nishimura Yuki Mitsufuji Alan Ritter Wei-ping Xu 50 0 0 07 Apr 2025
Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-Training Zhijun Wang Jiahuan Li Hao Zhou Rongxiang Weng J. Wang Xin Huang Xue Han Junlan Feng Chao Deng Shujian Huang LRM 48 1 0 02 Apr 2025
Sun-Shine: A Large Language Model for Tibetan Culture Cheng Huang Fan Gao Nyima Tashi Yutong Liu Xiangxiang Wang ... Gadeng Luosang Rinchen Dongrub Dorje Tashi Xiao Feng Yongbin Yu ALM 98 2 0 24 Mar 2025
TLUE: A Tibetan Language Understanding Evaluation Benchmark Fan Gao Cheng Huang Nyima Tashi Xiangxiang Wang Thupten Tsering ... Gadeng Luosang Rinchen Dongrub Dorje Tashi Xiao Feng Yongbin Yu ELM 74 2 0 15 Mar 2025
LAG-MMLU: Benchmarking Frontier LLM Understanding in Latvian and Giriama Naome A. Etori Kevin Lu Randu Karisa Arturs Kanepajs LRM ELM 98 0 0 14 Mar 2025
Every FLOP Counts: Scaling a 300B Mixture-of-Experts LING LLM without Premium GPUs Ling Team B. Zeng C. Huang Chao Zhang Changxin Tian ... Zhaoxin Huan Zujie Wen Zhenhang Sun Zhuoxuan Du Z. He MoE ALM 109 2 0 07 Mar 2025
Self-Memory Alignment: Mitigating Factual Hallucinations with Generalized Improvement Siyuan Zhang Y. Zhang Yinpeng Dong Hang Su HILM KELM 130 0 0 26 Feb 2025
MLaKE: Multilingual Knowledge Editing Benchmark for Large Language Models Zihao Wei Jingcheng Deng Liang Pang Hanxing Ding Huawei Shen Xueqi Cheng KELM 81 4 0 20 Feb 2025
Large Language Diffusion Models Shen Nie Fengqi Zhu Zebin You Xiaolu Zhang Jingyang Ou Jun Hu Jun Zhou Yankai Lin Ji-Rong Wen Chongxuan Li 100 12 0 14 Feb 2025
Improving Natural Language Understanding for LLMs via Large-Scale Instruction Synthesis Lin Yuan Jun Xu Honghao Gui Mengshu Sun Zhiqiang Zhang Lei Liang Jun Zhou AI4CE 153 0 0 06 Feb 2025
UGPhysics: A Comprehensive Benchmark for Undergraduate Physics Reasoning with Large Language Models Xin Xu Qiyun Xu Tong Xiao Tianhao Chen Yuchen Yan Jiaxin Zhang Shizhe Diao Can Yang Yang Wang ELM LRM AI4CE 100 2 0 01 Feb 2025
Baichuan-Omni-1.5 Technical Report Yadong Li J. Liu Tao Zhang Tao Zhang S. Chen ... Jianhua Xu Haoze Sun Mingan Lin Zenan Zhou Weipeng Chen AuLLM 70 10 0 28 Jan 2025
Activating Distributed Visual Region within LLMs for Efficient and Effective Vision-Language Training and Inference Siyuan Wang Dianyi Wang Chengxing Zhou Zejun Li Zhihao Fan Xuanjing Huang Zhongyu Wei VLM 129 0 0 17 Dec 2024
LLM-NEO: Parameter Efficient Knowledge Distillation for Large Language Models Runming Yang Taiqiang Wu Jiahao Wang Pengfei Hu Ngai Wong Yujiu Yang Yujiu Yang 95 0 0 11 Nov 2024
MoH: Multi-Head Attention as Mixture-of-Head Attention Peng Jin Bo Zhu Li Yuan Shuicheng Yan MoE 29 13 0 15 Oct 2024
Efficiently Democratizing Medical LLMs for 50 Languages via a Mixture of Language Family Experts Guorui Zheng Xidong Wang Juhao Liang Nuo Chen Yuping Zheng Benyou Wang MoE 30 5 0 14 Oct 2024
Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-training Gen Luo Xue Yang Wenhan Dou Zhaokai Wang Jifeng Dai Jifeng Dai Yu Qiao Xizhou Zhu VLM MLLM 62 25 0 10 Oct 2024
Determine-Then-Ensemble: Necessity of Top-k Union for Large Language Model Ensembling Yuxuan Yao Han Wu Mingyang Liu Sichun Luo Xiongwei Han Jie Liu Zhijiang Guo Linqi Song 56 4 0 03 Oct 2024
Enabling Real-Time Conversations with Minimal Training Costs Wang Xu Shuo Wang Weilin Zhao Xu Han Yukun Yan Yudi Zhang Zhe Tao Zhiyuan Liu Wanxiang Che 19 4 0 18 Sep 2024
Cracking the Code: Multi-domain LLM Evaluation on Real-World Professional Exams in Indonesia Fajri Koto ELM 42 2 0 13 Sep 2024
Training on the Benchmark Is Not All You Need Shiwen Ni Xiangtao Kong Chengming Li Xiping Hu Ruifeng Xu Jia Zhu Min Yang 56 5 0 03 Sep 2024
An Investigation of Warning Erroneous Chat Translations in Cross-lingual Communication Yunmeng Li Jun Suzuki Makoto Morishita Kaori Abe Kentaro Inui 53 1 0 28 Aug 2024
Too Late to Train, Too Early To Use? A Study on Necessity and Viability of Low-Resource Bengali LLMs Tamzeed Mahfuz Satak Kumar Dey Ruwad Naswan Hasnaen Adil Khondker Salman Sayeed Haz Sameen Shahgir 29 0 0 29 Jun 2024
The Music Maestro or The Musically Challenged, A Massive Music Evaluation Benchmark for Large Language Models Jiajia Li Lu Yang Mingni Tang Cong Chen Zuchao Li Ping Wang Hai Zhao LM&MA 34 4 0 22 Jun 2024
OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI Zhen Huang Zengzhi Wang Shijie Xia Xuefeng Li Haoyang Zou ... Yuxiang Zheng Shaoting Zhang Dahua Lin Yu Qiao Pengfei Liu ELM LRM 43 25 0 18 Jun 2024
The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models Seungone Kim Juyoung Suk Ji Yong Cho Shayne Longpre Chaeeun Kim ... Sean Welleck Graham Neubig Moontae Lee Kyungjae Lee Minjoon Seo ELM ALM LM&MA 97 29 0 09 Jun 2024
M4U: Evaluating Multilingual Understanding and Reasoning for Large Multimodal Models Hongyu Wang Jiayu Xu Senwei Xie Ruiping Wang Jialin Li Zhaojie Xie Bin Zhang Chuyan Xiong Xilin Chen ELM VLM LRM 86 7 0 24 May 2024
LHMKE: A Large-scale Holistic Multi-subject Knowledge Evaluation Benchmark for Chinese Large Language Models Chuang Liu Renren Jin Yuqi Ren Deyi Xiong ELM 30 0 0 19 Mar 2024
Yi: Open Foundation Models by 01.AI 01. AI Alex Young 01.AI Alex Young Bei Chen Chao Li ... Yue Wang Yuxuan Cai Zhenyu Gu Zhiyuan Liu Zonghong Dai OSLM LRM 121 495 0 07 Mar 2024
Apollo: A Lightweight Multilingual Medical LLM towards Democratizing Medical AI to 6B People Xidong Wang Nuo Chen Junying Chen Yan Hu Yidong Wang Xiangbo Wu Anningzhe Gao Xiang Wan Haizhou Li Benyou Wang LM&MA 33 25 0 06 Mar 2024
Enabling Weak LLMs to Judge Response Reliability via Meta Ranking Zijun Liu Boqun Kou Peng Li Ming Yan Ji Zhang Fei Huang Yang Janet Liu 24 2 0 19 Feb 2024
Learning to Trust Your Feelings: Leveraging Self-awareness in LLMs for Hallucination Mitigation Yuxin Liang Zhuoyang Song Hao Wang Jiaxing Zhang HILM 31 28 0 27 Jan 2024
HuatuoGPT-II, One-stage Training for Medical Adaption of LLMs Junying Chen Xidong Wang Anningzhe Gao Feng Jiang Shunian Chen ... Chuyi Kong Jianquan Li Xiang Wan Haizhou Li Benyou Wang LM&MA 24 61 0 16 Nov 2023
ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferences Yuanhe Tian Ruyi Gan Yan Song Jiaxing Zhang Yongdong Zhang AI4MH AI4CE LM&MA 24 30 0 10 Nov 2023
PromptCBLUE: A Chinese Prompt Tuning Benchmark for the Medical Domain Wei-wei Zhu Xiaoling Wang Huanran Zheng Mosha Chen Buzhou Tang ELM LM&MA 21 33 0 22 Oct 2023
LawBench: Benchmarking Legal Knowledge of Large Language Models Zhiwei Fei Xiaoyu Shen D. Zhu Fengzhe Zhou Zhuo Han Songyang Zhang Kai-xiang Chen Zongwen Shen Jidong Ge ELM AILaw 24 33 0 28 Sep 2023
Baichuan 2: Open Large-scale Language Models Ai Ming Yang Bin Xiao Bingning Wang Borong Zhang Ce Bian ... Youxin Jiang Yuchen Gao Yupeng Zhang Zenan Zhou Zhiying Wu ELM LRM 66 699 0 19 Sep 2023
LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions Minghao Wu Abdul Waheed Chiyu Zhang Muhammad Abdul-Mageed Alham Fikri Aji ALM 127 118 0 27 Apr 2023
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng-Zhen Zhang Yuxiao Dong Jie Tang BDL LRM 245 1,071 0 05 Oct 2022
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,943 0 20 Apr 2018
Teaching Machines to Read and Comprehend Karl Moritz Hermann Tomás Kociský Edward Grefenstette L. Espeholt W. Kay Mustafa Suleyman Phil Blunsom 170 3,508 0 10 Jun 2015