Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
2306.09212
Cited By
v1
v2 (latest)
CMMLU: Measuring massive multitask language understanding in Chinese
Annual Meeting of the Association for Computational Linguistics (ACL), 2023
15 June 2023
Jinyan Su
Yixuan Zhang
Fajri Koto
Yifei Yang
Hai Zhao
Yeyun Gong
Nan Duan
Tim Baldwin
ALM
ELM
Re-assign community
ArXiv (abs)
PDF
HTML
Papers citing
"CMMLU: Measuring massive multitask language understanding in Chinese"
50 / 267 papers shown
Title
OmChat: A Recipe to Train Multimodal Language Models with Strong Long Context and Video Understanding
Tiancheng Zhao
Qianqian Zhang
Kyusong Lee
Peng Liu
Lu Zhang
Chunxin Fang
Jiajia Liao
Kelei Jiang
Yibo Ma
Ruochen Xu
MLLM
VLM
215
8
0
06 Jul 2024
CFinBench: A Comprehensive Chinese Financial Benchmark for Large Language Models
Ying Nie
Binwei Yan
Tianyu Guo
Hao Liu
Haoyu Wang
...
Weihao Wang
Qiang Li
Weijian Sun
Yunhe Wang
Dacheng Tao
ELM
306
5
0
02 Jul 2024
Too Late to Train, Too Early To Use? A Study on Necessity and Viability of Low-Resource Bengali LLMs
Tamzeed Mahfuz
Satak Kumar Dey
Ruwad Naswan
Hasnaen Adil
Khondker Salman Sayeed
Haz Sameen Shahgir
209
4
0
29 Jun 2024
IRCAN: Mitigating Knowledge Conflicts in LLM Generation via Identifying and Reweighting Context-Aware Neurons
Dan Shi
Renren Jin
Shangda Wu
Weilong Dong
Xinwei Wu
Deyi Xiong
209
25
0
26 Jun 2024
MedCare: Advancing Medical LLMs through Decoupling Clinical Alignment and Knowledge Aggregation
Yusheng Liao
Shuyang Jiang
Yanfeng Wang
Yu Wang
238
4
0
25 Jun 2024
CaLMQA: Exploring culturally specific long-form question answering across 23 languages
Shane Arora
Marzena Karpinska
Hung-Ting Chen
Ipsita Bhattacharjee
Mohit Iyyer
Eunsol Choi
HILM
409
22
0
25 Jun 2024
The Music Maestro or The Musically Challenged, A Massive Music Evaluation Benchmark for Large Language Models
Jiajia Li
Lu Yang
Mingni Tang
Cong Chen
Zuchao Li
Ping Wang
Hai Zhao
LM&MA
179
10
0
22 Jun 2024
Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Models
Xinrong Zhang
Yingfa Chen
Shengding Hu
Xu Han
Zihang Xu
Yuanwei Xu
Weilin Zhao
Maosong Sun
Zhiyuan Liu
162
19
0
22 Jun 2024
InternLM-Law: An Open Source Chinese Legal Large Language Model
Zhiwei Fei
Songyang Zhang
Xiaoyu Shen
Dawei Zhu
Xiao Wang
...
Yining Li
Wenwei Zhang
Dahua Lin
Kai Chen
Jidong Ge
ELM
AILaw
281
4
0
21 Jun 2024
Is It Good Data for Multilingual Instruction Tuning or Just Bad Multilingual Evaluation for Large Language Models?
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024
Pinzhen Chen
Simon Yu
Zhicheng Guo
Barry Haddow
ELM
319
3
0
18 Jun 2024
OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI
Zhen Huang
Zengzhi Wang
Shijie Xia
Xuefeng Li
Haoyang Zou
...
Yuxiang Zheng
Shaoting Zhang
Dahua Lin
Yu Qiao
Pengfei Liu
ELM
LRM
257
68
0
18 Jun 2024
DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence
DeepSeek-AI
Qihao Zhu
Daya Guo
Zhihong Shao
Dejian Yang
...
Jiashi Li
Chenggang Zhao
Chong Ruan
Fuli Luo
Wenfeng Liang
MoE
LRM
ELM
VLM
236
344
0
17 Jun 2024
Evaluating the Generalization Ability of Quantized LLMs: Benchmark, Analysis, and Toolbox
Yijun Liu
Yuan Meng
Fang Wu
Shenhao Peng
Hang Yao
Chaoyu Guan
Chen Tang
Cheng Wang
Zhi Wang
Wenwu Zhu
MQ
271
9
0
15 Jun 2024
GEB-1.3B: Open Lightweight Large Language Model
Jie Wu
Yufeng Zhu
Lei Shen
Xuqing Lu
ALM
89
1
0
14 Jun 2024
ME-Switch: A Memory-Efficient Expert Switching Framework for Large Language Models
Jing Liu
Yazhe Niu
Mingyang Zhang
Yefei He
Jianfei Cai
Bohan Zhuang
MoE
108
2
0
13 Jun 2024
SciKnowEval: Evaluating Multi-level Scientific Knowledge of Large Language Models
Kehua Feng
Keyan Ding
Weijie Wang
Xiang Zhuang
Yuqi Tang
Ming Qin
Yu Zhao
ELM
289
12
0
13 Jun 2024
MultiPragEval: Multilingual Pragmatic Evaluation of Large Language Models
Dojun Park
Jiwoo Lee
Seohyun Park
Hyeyun Jeong
Youngeun Koo
Soonha Hwang
Seonwoo Park
Sungeun Lee
ELM
101
4
0
11 Jun 2024
BertaQA: How Much Do Language Models Know About Local Culture?
Julen Etxaniz
Gorka Azkune
A. Soroa
Oier López de Lacalle
Mikel Artetxe
188
17
0
11 Jun 2024
The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models
North American Chapter of the Association for Computational Linguistics (NAACL), 2024
Seungone Kim
Juyoung Suk
Ji Yong Cho
Shayne Longpre
Chaeeun Kim
...
Sean Welleck
Graham Neubig
Moontae Lee
Kyungjae Lee
Minjoon Seo
ELM
ALM
LM&MA
375
68
0
09 Jun 2024
WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild
International Conference on Learning Representations (ICLR), 2024
Bill Yuchen Lin
Yuntian Deng
Khyathi Chandu
Faeze Brahman
Abhilasha Ravichander
Valentina Pyatkin
Nouha Dziri
Ronan Le Bras
Yejin Choi
224
128
0
07 Jun 2024
Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Art
Chen Cecilia Liu
Iryna Gurevych
Anna Korhonen
498
14
0
06 Jun 2024
Wings: Learning Multimodal LLMs without Text-only Forgetting
Yi-Kai Zhang
Shiyin Lu
Yang Li
Yanqing Ma
Qing-Guo Chen
Zhao Xu
Weihua Luo
Kaifu Zhang
De-Chuan Zhan
Han-Jia Ye
VLM
260
17
0
05 Jun 2024
Sparsity-Accelerated Training for Large Language Models
Da Ma
Lu Chen
Pengyu Wang
Hongshen Xu
Hanqi Li
Liangtai Sun
Su Zhu
Shuai Fan
Kai Yu
LRM
122
2
0
03 Jun 2024
Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
Tianwen Wei
Bo Zhu
Liang Zhao
Cheng Cheng
Biye Li
...
Yutuan Ma
Rui Hu
Shuicheng Yan
Han Fang
Yahui Zhou
MoE
257
49
0
03 Jun 2024
MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Series
Ge Zhang
Scott Qu
Jiaheng Liu
Chenchen Zhang
Chenghua Lin
...
Zi-Kai Zhao
Jiajun Zhang
Wanli Ouyang
Wenhao Huang
Lei Ma
ELM
250
66
0
29 May 2024
IAPT: Instruction-Aware Prompt Tuning for Large Language Models
Wei-wei Zhu
Aaron Xuxiang Tian
Congrui Yin
Yuan Ni
Xiaoling Wang
Guotong Xie
267
0
0
28 May 2024
M4U: Evaluating Multilingual Understanding and Reasoning for Large Multimodal Models
Hongyu Wang
Jiayu Xu
Senwei Xie
Ruiping Wang
Jialin Li
Zhaojie Xie
Bin Zhang
Chuyan Xiong
Xilin Chen
ELM
VLM
LRM
355
9
0
24 May 2024
360Zhinao Technical Report
360Zhinao Team
185
0
0
22 May 2024
Medical Dialogue: A Survey of Categories, Methods, Evaluation and Challenges
Xiaoming Shi
Zeming Liu
Li Du
Yuxuan Wang
Hongru Wang
Yuhang Guo
Tong Ruan
Jie Xu
Shaoting Zhang
LM&MA
ELM
302
8
0
17 May 2024
Dynamic data sampler for cross-language transfer learning in large language models
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2024
Yudong Li
Yuhao Feng
Wen Zhou
Zhe Zhao
Linlin Shen
Cheng-An Hou
Xianxu Hou
166
5
0
17 May 2024
Benchmarking Large Language Models on CFLUE -- A Chinese Financial Language Understanding Evaluation Dataset
Annual Meeting of the Association for Computational Linguistics (ACL), 2024
Jie Zhu
Junhui Li
Yalong Wen
Lifan Guo
ELM
ALM
136
17
0
17 May 2024
CPsyExam: A Chinese Benchmark for Evaluating Psychology using Examinations
International Conference on Computational Linguistics (COLING), 2024
Jiahao Zhao
Jingwei Zhu
Minghuan Tan
Min Yang
Di Yang
Chenhao Zhang
Guancheng Ye
Chengming Li
Xiping Hu
ELM
250
1
0
16 May 2024
OpenBA-V2: Reaching 77.3% High Compression Ratio with Fast Multi-Stage Pruning
Dan Qiao
Yi Su
Pinzheng Wang
Jing Ye
Wen Xie
...
Wenliang Chen
Guohong Fu
Guodong Zhou
Qiaoming Zhu
Min Zhang
MQ
202
1
0
09 May 2024
ChuXin: 1.6B Technical Report
Xiaomin Zhuang
Yufan Jiang
Qiaozhi He
Zhihua Wu
ALM
159
0
0
08 May 2024
CourseGPT-zh: an Educational Large Language Model Based on Knowledge Distillation Incorporating Prompt Optimization
Zheyan Qu
Lu Yin
Zitong Yu
Wenbo Wang
Xing Zhang
ALM
127
6
0
08 May 2024
DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
DeepSeek-AI
Aixin Liu
Bei Feng
Bin Wang
Bingxuan Wang
...
Zhuoshu Li
Zihan Wang
Zihui Gu
Zilin Li
Ziwei Xie
MoE
392
882
0
07 May 2024
Do Large Language Models Understand Conversational Implicature -- A case study with a chinese sitcom
Shisen Yue
Siyuan Song
Xinyuan Cheng
Hai Hu
205
11
0
30 Apr 2024
Benchmarking Benchmark Leakage in Large Language Models
Ruijie Xu
Zengzhi Wang
Run-Ze Fan
Pengfei Liu
211
89
0
29 Apr 2024
FoundaBench: Evaluating Chinese Fundamental Knowledge Capabilities of Large Language Models
Wei Li
Ren Ma
Jiang Wu
Chenya Gu
Jiahui Peng
Jinyang Len
Songyang Zhang
Hang Yan
Dahua Lin
Conghui He
ELM
138
1
0
29 Apr 2024
Continual Learning of Large Language Models: A Comprehensive Survey
Haizhou Shi
Zihao Xu
Hengyi Wang
Weiyi Qin
Wenyuan Wang
Yibin Wang
Zifeng Wang
Sayna Ebrahimi
Hao Wang
CLL
KELM
LRM
321
145
0
25 Apr 2024
Tele-FLM Technical Report
Xiang Li
Yiqun Yao
Xin Jiang
Xuezhi Fang
Chao Wang
...
Yequan Wang
Zhongjiang He
Zhongyuan Wang
Xuelong Li
Tiejun Huang
189
10
0
25 Apr 2024
MING-MOE: Enhancing Medical Multi-Task Learning in Large Language Models with Sparse Mixture of Low-Rank Adapter Experts
Yusheng Liao
Shuyang Jiang
Yu Wang
Yanfeng Wang
MoE
210
10
0
13 Apr 2024
Khayyam Challenge (PersianMMLU): Is Your LLM Truly Wise to The Persian Language?
Omid Ghahroodi
Marzia Nouri
Mohammad V. Sanian
Alireza Sahebi
D. Dastgheib
Ehsaneddin Asgari
M. Baghshah
M. Rohban
ELM
AAML
136
22
0
09 Apr 2024
Chinese Tiny LLM: Pretraining a Chinese-Centric Large Language Model
Xinrun Du
Zhouliang Yu
Songyang Gao
Ding Pan
Yuyang Cheng
...
Tianyu Zheng
Xinchen Luo
Guorui Zhou
Lei Ma
Ge Zhang
271
26
0
05 Apr 2024
IndoCulture: Exploring Geographically-Influenced Cultural Commonsense Reasoning Across Eleven Indonesian Provinces
Transactions of the Association for Computational Linguistics (TACL), 2024
Fajri Koto
Rahmad Mahendra
Nurul Aisyah
Timothy Baldwin
LRM
295
34
0
02 Apr 2024
Measuring Taiwanese Mandarin Language Understanding
Po-Heng Chen
Sijia Cheng
Wei-Lin Chen
Yen-Ting Lin
Yun-Nung Chen
ELM
223
2
0
29 Mar 2024
Checkpoint Merging via Bayesian Optimization in LLM Pretraining
Deyuan Liu
Zecheng Wang
Bingning Wang
Weipeng Chen
Chunshan Li
Zhiying Tu
Dianhui Chu
Bo Li
Dianbo Sui
MoMe
239
26
0
28 Mar 2024
Large Language Models for Education: A Survey and Outlook
Shen Wang
Tianlong Xu
Hang Li
Chaoli Zhang
Joleen Liang
Shucheng Zhou
Philip S. Yu
Qingsong Wen
AI4Ed
314
211
0
26 Mar 2024
COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning
Yuelin Bai
Xinrun Du
Yiming Liang
Yonggang Jin
Ziqiang Liu
...
Chenghua Lin
Jie Fu
Min Yang
Shiwen Ni
Ge Zhang
ALM
128
50
0
26 Mar 2024
Can multiple-choice questions really be useful in detecting the abilities of LLMs?
Wangyue Li
Liangzhi Li
Tong Xiang
Xiao Liu
Wei Deng
Noa Garcia
ELM
364
66
0
26 Mar 2024
Previous
1
2
3
4
5
6
Next