Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
2009.03300
Cited By
v1
v2
v3 (latest)
Measuring Massive Multitask Language Understanding
International Conference on Learning Representations (ICLR), 2020
7 September 2020
Dan Hendrycks
Collin Burns
Steven Basart
Andy Zou
Mantas Mazeika
Basel Alomair
Jacob Steinhardt
ELM
RALM
Re-assign community
ArXiv (abs)
PDF
HTML
HuggingFace (3 upvotes)
Papers citing
"Measuring Massive Multitask Language Understanding"
50 / 4,481 papers shown
INSTRUCTEVAL: Towards Holistic Evaluation of Instruction-Tuned Large Language Models
Yew Ken Chia
Pengfei Hong
Lidong Bing
Soujanya Poria
ELM
256
75
0
07 Jun 2023
How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources
Neural Information Processing Systems (NeurIPS), 2023
Yizhong Wang
Michal Guerquin
Pradeep Dasigi
Jack Hessel
Tushar Khot
...
Aman Rangapur
Kelsey MacMillan
Noah A. Smith
Iz Beltagy
Hannaneh Hajishirzi
ALM
ELM
352
469
0
07 Jun 2023
PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts
Lingyao Li
Yongfeng Zhang
Jiaheng Zhou
Zichen Wang
Hao Chen
...
Linyi Yang
Weirong Ye
Yue Zhang
Neil Zhenqiang Gong
Xingxu Xie
SILM
430
211
0
07 Jun 2023
Benchmarking Foundation Models with Language-Model-as-an-Examiner
Neural Information Processing Systems (NeurIPS), 2023
Yushi Bai
Jiahao Ying
Yixin Cao
Xin Lv
Yuze He
...
Yijia Xiao
Haozhe Lyu
Jiayin Zhang
Juanzi Li
Lei Hou
ALM
ELM
269
199
0
07 Jun 2023
The Emergence of Essential Sparsity in Large Pre-trained Models: The Weights that Matter
Neural Information Processing Systems (NeurIPS), 2023
Ajay Jaiswal
Shiwei Liu
Tianlong Chen
Zinan Lin
VLM
276
44
0
06 Jun 2023
Applying Standards to Advance Upstream & Downstream Ethics in Large Language Models
Jose Berengueres
Marybeth Sandell
181
0
0
06 Jun 2023
Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
Neural Information Processing Systems (NeurIPS), 2023
Kenneth Li
Oam Patel
Fernanda Viégas
Hanspeter Pfister
Martin Wattenberg
KELM
HILM
746
833
0
06 Jun 2023
Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Dataset
Neural Information Processing Systems (NeurIPS), 2023
Junling Liu
Peilin Zhou
Yining Hua
Dading Chong
Zhongyu Tian
...
Helin Wang
Chenyu You
Zhenhua Guo
Lei Zhu
Michael Lingzhi Li
LM&MA
ELM
493
115
0
05 Jun 2023
MultiLegalPile: A 689GB Multilingual Legal Corpus
Annual Meeting of the Association for Computational Linguistics (ACL), 2023
Joel Niklaus
Veton Matoshi
Matthias Sturmer
Ilias Chalkidis
Daniel E. Ho
AILaw
ELM
422
61
0
03 Jun 2023
Reimagining Retrieval Augmented Language Models for Answering Queries
Annual Meeting of the Association for Computational Linguistics (ACL), 2023
W. Tan
Yuliang Li
Pedro Rodriguez
Rich James
Xi Lin
A. Halevy
Scott Yih
KELM
LRM
305
13
0
01 Jun 2023
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets
Annual Meeting of the Association for Computational Linguistics (ACL), 2023
Md Tahmid Rahman Laskar
M Saiful Bari
Mizanur Rahman
Md Amran Hossen Bhuiyan
Shafiq Joty
J. Huang
LM&MA
ELM
ALM
500
212
0
29 May 2023
LLM-QAT: Data-Free Quantization Aware Training for Large Language Models
Annual Meeting of the Association for Computational Linguistics (ACL), 2023
Zechun Liu
Barlas Oğuz
Changsheng Zhao
Ernie Chang
Pierre Stock
Yashar Mehdad
Yangyang Shi
Raghuraman Krishnamoorthi
Vikas Chandra
MQ
263
294
0
29 May 2023
Conformal Prediction with Large Language Models for Multi-Choice Question Answering
Bhawesh Kumar
Cha-Chen Lu
Gauri Gupta
Anil Palepu
David R. Bellamy
Ramesh Raskar
Andrew L. Beam
442
101
0
28 May 2023
What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks
Neural Information Processing Systems (NeurIPS), 2023
Taicheng Guo
Kehan Guo
B. Nan
Zhengwen Liang
Zhichun Guo
Nitesh Chawla
Olaf Wiest
Xiangliang Zhang
ELM
518
210
0
27 May 2023
Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In
Annual Meeting of the Association for Computational Linguistics (ACL), 2023
Zichun Yu
Chenyan Xiong
S. Yu
Zhiyuan Liu
KELM
VLM
289
83
0
27 May 2023
Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models
Annual Meeting of the Association for Computational Linguistics (ACL), 2023
Yuhui Zhang
Michihiro Yasunaga
Zhengping Zhou
Jeff Z. HaoChen
James Zou
Abigail Z. Jacobs
Serena Yeung
265
11
0
27 May 2023
Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance
Yao Fu
Litu Ou
Mingyu Chen
Yuhao Wan
Hao-Chun Peng
Tushar Khot
LLMAG
ELM
LRM
ReLM
212
125
0
26 May 2023
Training Socially Aligned Language Models on Simulated Social Interactions
International Conference on Learning Representations (ICLR), 2023
Ruibo Liu
Ruixin Yang
Chenyan Jia
Ge Zhang
Denny Zhou
Andrew M. Dai
Diyi Yang
Soroush Vosoughi
ALM
285
88
0
26 May 2023
Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation
International Conference on Learning Representations (ICLR), 2023
Niels Mündler
Jingxuan He
Slobodan Jenko
Martin Vechev
HILM
308
156
0
25 May 2023
The False Promise of Imitating Proprietary LLMs
Arnav Gudibande
Eric Wallace
Charles Burton Snell
Xinyang Geng
Hao Liu
Pieter Abbeel
Sergey Levine
Dawn Song
ALM
331
250
0
25 May 2023
On Degrees of Freedom in Defining and Testing Natural Language Understanding
Annual Meeting of the Association for Computational Linguistics (ACL), 2023
Saku Sugawara
S. Tsugita
ELM
323
2
0
24 May 2023
C-STS: Conditional Semantic Textual Similarity
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Ameet Deshpande
Carlos E. Jimenez
Howard Chen
Vishvak Murahari
Victoria Graf
Tanmay Rajpurohit
Ashwin Kalyan
Danqi Chen
Karthik Narasimhan
182
3
0
24 May 2023
Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Daman Arora
H. Singh
Mausam
ELM
LRM
412
75
0
24 May 2023
The Art of SOCRATIC QUESTIONING: Recursive Thinking with Large Language Models
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Jingyuan Qi
Zhiyang Xu
Ying Shen
Minqian Liu
dingnan jin
Qifan Wang
Lifu Huang
ReLM
LRM
KELM
152
21
0
24 May 2023
How Predictable Are Large Language Model Capabilities? A Case Study on BIG-bench
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Qinyuan Ye
Harvey Yiyun Fu
Xiang Ren
Robin Jia
ELM
270
34
0
24 May 2023
In-Context Impersonation Reveals Large Language Models' Strengths and Biases
Neural Information Processing Systems (NeurIPS), 2023
Leonard Salewski
Stephan Alaniz
Isabel Rio-Torto
Eric Schulz
Zeynep Akata
305
181
0
24 May 2023
Estimating Large Language Model Capabilities without Labeled Test Data
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Harvey Yiyun Fu
Qinyuan Ye
Albert Xu
Xiang Ren
Robin Jia
268
10
0
24 May 2023
Mixture-of-Experts Meets Instruction Tuning:A Winning Combination for Large Language Models
International Conference on Learning Representations (ICLR), 2023
Sheng Shen
Le Hou
Yan-Quan Zhou
Nan Du
Shayne Longpre
...
Vincent Zhao
Hongkun Yu
Kurt Keutzer
Trevor Darrell
Denny Zhou
ALM
MoE
442
83
0
24 May 2023
Emergent inabilities? Inverse scaling over the course of pretraining
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
J. Michaelov
Benjamin Bergen
LRM
ReLM
183
6
0
24 May 2023
Increasing Probability Mass on Answer Choices Does Not Always Improve Accuracy
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Sarah Wiegreffe
Matthew Finlayson
Oyvind Tafjord
Peter Clark
Ashish Sabharwal
191
10
0
24 May 2023
Sources of Hallucination by Large Language Models on Inference Tasks
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Nick McKenna
Tianyi Li
Liang Cheng
Mohammad Javad Hosseini
Mark Johnson
Mark Steedman
LRM
HILM
293
243
0
23 May 2023
RetICL: Sequential Retrieval of In-Context Examples with Reinforcement Learning
Alexander Scarlatos
Andrew Lan
OffRL
LRM
260
28
0
23 May 2023
Improving Factuality and Reasoning in Language Models through Multiagent Debate
International Conference on Machine Learning (ICML), 2023
Yilun Du
Shuang Li
Antonio Torralba
J. Tenenbaum
Igor Mordatch
LLMAG
LRM
351
1,182
0
23 May 2023
QLoRA: Efficient Finetuning of Quantized LLMs
Neural Information Processing Systems (NeurIPS), 2023
Tim Dettmers
Artidoro Pagnoni
Ari Holtzman
Luke Zettlemoyer
ALM
616
3,684
0
23 May 2023
Query Rewriting for Retrieval-Augmented Large Language Models
Xinbei Ma
Yeyun Gong
Pengcheng He
Hai Zhao
Nan Duan
KELM
LRM
236
192
0
23 May 2023
Enhancing Chat Language Models by Scaling High-quality Instructional Conversations
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Ning Ding
Yulin Chen
Bokai Xu
Yujia Qin
Zhi Zheng
Shengding Hu
Zhiyuan Liu
Maosong Sun
Bowen Zhou
ALM
365
747
0
23 May 2023
Skill-Based Few-Shot Selection for In-Context Learning
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Shengnan An
Bo Zhou
Zeqi Lin
Qiang Fu
B. Chen
Nanning Zheng
Weizhu Chen
Jian-Guang Lou
398
43
0
23 May 2023
Memory-Efficient Fine-Tuning of Compressed Large Language Models via sub-4-bit Integer Quantization
Neural Information Processing Systems (NeurIPS), 2023
Jeonghoon Kim
J. H. Lee
Sungdong Kim
Joonsuk Park
Kang Min Yoo
S. Kwon
Dongsoo Lee
MQ
374
131
0
23 May 2023
Can Large Language Models Capture Dissenting Human Voices?
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Noah Lee
Na Min An
James Thorne
ALM
319
41
0
23 May 2023
Aligning Large Language Models through Synthetic Feedback
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Sungdong Kim
Sanghwan Bae
Jamin Shin
Soyoung Kang
Donghyun Kwak
Kang Min Yoo
Minjoon Seo
ALM
SyDa
273
84
0
23 May 2023
Exploring Self-supervised Logic-enhanced Training for Large Language Models
North American Chapter of the Association for Computational Linguistics (NAACL), 2023
Fangkai Jiao
Zhiyang Teng
Bosheng Ding
Zhengyuan Liu
Nancy F. Chen
Shafiq Joty
ReLM
LRM
245
8
0
23 May 2023
Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models
Annual Meeting of the Association for Computational Linguistics (ACL), 2023
Alfonso Amayuelas
Kyle Wong
Liangming Pan
Wenhu Chen
Wenjie Wang
398
40
0
23 May 2023
Polyglot or Not? Measuring Multilingual Encyclopedic Knowledge in Foundation Models
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Tim Schott
Daniel Furman
Shreshta Bhat
ELM
283
5
0
23 May 2023
CLASS: A Design Framework for building Intelligent Tutoring Systems based on Learning Science principles
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Shashank Sonkar
Lucy Liu
D. B. Mallick
Richard G. Baraniuk
309
60
0
22 May 2023
Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sources
International Conference on Learning Representations (ICLR), 2023
Xingxuan Li
Ruochen Zhao
Yew Ken Chia
Bosheng Ding
Shafiq Joty
Soujanya Poria
Lidong Bing
HILM
BDL
LRM
468
144
0
22 May 2023
Should We Attend More or Less? Modulating Attention for Fairness
A. Zayed
Gonçalo Mordido
Samira Shabanian
Sarath Chandar
264
15
0
22 May 2023
RWKV: Reinventing RNNs for the Transformer Era
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Bo Peng
Eric Alcaide
Quentin G. Anthony
Alon Albalak
Samuel Arcadinho
...
Qihang Zhao
P. Zhou
Qinghua Zhou
Jian Zhu
Rui-Jie Zhu
579
856
0
22 May 2023
Iterative Forward Tuning Boosts In-Context Learning in Language Models
Annual Meeting of the Association for Computational Linguistics (ACL), 2023
Jiaxi Yang
Binyuan Hui
Min Yang
Bailin Wang
Bowen Li
Binhua Li
Fei Huang
Yongbin Li
267
19
0
22 May 2023
ExplainCPE: A Free-text Explanation Benchmark of Chinese Pharmacist Examination
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Dongfang Li
Jindi Yu
Baotian Hu
Zhenran Xu
Hao Fei
ELM
178
14
0
22 May 2023
Meta-in-context learning in large language models
Neural Information Processing Systems (NeurIPS), 2023
Julian Coda-Forno
Marcel Binz
Zeynep Akata
M. Botvinick
Jane X. Wang
Eric Schulz
LRM
429
60
0
22 May 2023
Previous
1
2
3
...
86
87
88
89
90
Next
