Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
2009.03300
Cited By
v1
v2
v3 (latest)
Measuring Massive Multitask Language Understanding
International Conference on Learning Representations (ICLR), 2020
7 September 2020
Dan Hendrycks
Collin Burns
Steven Basart
Andy Zou
Mantas Mazeika
Basel Alomair
Jacob Steinhardt
ELM
RALM
Re-assign community
ArXiv (abs)
PDF
HTML
HuggingFace (3 upvotes)
Papers citing
"Measuring Massive Multitask Language Understanding"
50 / 4,486 papers shown
ART: Automatic multi-step reasoning and tool-use for large language models
Bhargavi Paranjape
Scott M. Lundberg
Sameer Singh
Hannaneh Hajishirzi
Luke Zettlemoyer
Marco Tulio Ribeiro
KELM
ReLM
LRM
320
192
0
16 Mar 2023
The Learnability of In-Context Learning
Neural Information Processing Systems (NeurIPS), 2023
Noam Wies
Yoav Levine
Amnon Shashua
317
159
0
14 Mar 2023
Generating multiple-choice questions for medical question answering with distractors and cue-masking
International Conference on Language Resources and Evaluation (LREC), 2023
Damien Sileo
Kanimozhi Uma
Marie-Francine Moens
208
5
0
13 Mar 2023
LLaMA: Open and Efficient Foundation Language Models
Hugo Touvron
Thibaut Lavril
Gautier Izacard
Xavier Martinet
Marie-Anne Lachaux
...
Faisal Azhar
Aurelien Rodriguez
Armand Joulin
Edouard Grave
Guillaume Lample
ALM
PILM
8.6K
18,046
0
27 Feb 2023
Testing AI on language comprehension tasks reveals insensitivity to underlying meaning
Scientific Reports (Sci Rep), 2023
Vittoria Dentella
Fritz Guenther
Elliot Murphy
G. Marcus
Evelina Leivada
ELM
432
52
0
23 Feb 2023
Complex QA and language models hybrid architectures, Survey
Xavier Daull
P. Bellot
Emmanuel Bruno
Vincent Martin
Elisabeth Murisasco
ELM
731
17
0
17 Feb 2023
Augmented Language Models: a Survey
Grégoire Mialon
Roberto Dessì
Maria Lomeli
Christoforos Nalmpantis
Ramakanth Pasunuru
...
Jane Dwivedi-Yu
Asli Celikyilmaz
Edouard Grave
Yann LeCun
Thomas Scialom
LRM
KELM
296
497
0
15 Feb 2023
STREET: A Multi-Task Structured Reasoning and Explanation Benchmark
International Conference on Learning Representations (ICLR), 2023
D. Ribeiro
Shen Wang
Xiaofei Ma
He Zhu
Rui Dong
...
William Yang Wang
Zhiheng Huang
George Karypis
Bing Xiang
Dan Roth
LRM
ReLM
170
26
0
13 Feb 2023
Can GPT-3 Perform Statutory Reasoning?
International Conference on Artificial Intelligence and Law (ICAIL), 2023
Andrew Blair-Stanek
Nils Holzenberger
Benjamin Van Durme
ELM
LRM
323
124
0
13 Feb 2023
Mathematical Capabilities of ChatGPT
Neural Information Processing Systems (NeurIPS), 2023
Simon Frieder
Luca Pinchetti
Alexis Chevalier
Ryan-Rhys Griffiths
Tommaso Salvatori
Thomas Lukasiewicz
P. Petersen
Julius Berner
ELM
AI4MH
513
530
0
31 Jan 2023
The Flan Collection: Designing Data and Methods for Effective Instruction Tuning
International Conference on Machine Learning (ICML), 2023
Shayne Longpre
Le Hou
Tu Vu
Albert Webson
Hyung Won Chung
...
Denny Zhou
Quoc V. Le
Barret Zoph
Jason W. Wei
Adam Roberts
ALM
444
853
0
31 Jan 2023
LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domain
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Joel Niklaus
Veton Matoshi
Pooja Rani
Andrea Galassi
Matthias Sturmer
Ilias Chalkidis
ELM
AILaw
385
81
0
30 Jan 2023
REPLUG: Retrieval-Augmented Black-Box Language Models
North American Chapter of the Association for Computational Linguistics (NAACL), 2023
Weijia Shi
Sewon Min
Michihiro Yasunaga
Minjoon Seo
Rich James
M. Lewis
Luke Zettlemoyer
Anuj Kumar
RALM
VLM
KELM
729
866
0
30 Jan 2023
ThoughtSource: A central hub for large language model reasoning data
Scientific Data (Sci Data), 2023
Simon Ott
Konstantin Hebenstreit
Valentin Liévin
C. Hother
M. Moradi
Maximilian Mayrhauser
Robert Praas
Ole Winther
Matthias Samwald
ReLM
LRM
533
60
0
27 Jan 2023
MAUD: An Expert-Annotated Legal NLP Dataset for Merger Agreement Understanding
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Steven H. Wang
Antoine Scardigli
Leonard Tang
Wei Chen
D.M. Levkin
Anya Chen
Spencer Ball
Thomas Woodside
Oliver Zhang
Dan Hendrycks
AILaw
ELM
209
37
0
02 Jan 2023
Inconsistencies in Masked Language Models
Tom Young
Yunan Chen
Yang You
288
2
0
30 Dec 2022
Large Language Models Encode Clinical Knowledge
Nature (Nature), 2022
K. Singhal
Shekoofeh Azizi
T. Tu
S. S. Mahdavi
Jason W. Wei
...
A. Rajkomar
Joelle Barral
Christopher Semturs
Alan Karthikesalingam
Vivek Natarajan
LM&MA
ELM
AI4MH
608
3,513
0
26 Dec 2022
Quality at the Tail of Machine Learning Inference
Zhengxin Yang
Wanling Gao
Chunjie Luo
Lei Wang
Fei Tang
Xu Wen
Jianfeng Zhan
198
1
0
25 Dec 2022
OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization
Srinivasan Iyer
Xi Lin
Ramakanth Pasunuru
Todor Mihaylov
Daniel Simig
...
Jeff Wang
Christopher Dewan
Asli Celikyilmaz
Luke Zettlemoyer
Veselin Stoyanov
ALM
487
303
0
22 Dec 2022
ORCA: A Challenging Benchmark for Arabic Language Understanding
Annual Meeting of the Association for Computational Linguistics (ACL), 2022
AbdelRahim Elmadany
El Moatez Billah Nagoudi
Muhammad Abdul-Mageed
ELM
301
60
0
21 Dec 2022
A Survey of Deep Learning for Mathematical Reasoning
Annual Meeting of the Association for Computational Linguistics (ACL), 2022
Pan Lu
Liang Qiu
Wenhao Yu
Sean Welleck
Kai-Wei Chang
ReLM
LRM
289
184
0
20 Dec 2022
Evaluating Human-Language Model Interaction
Mina Lee
Megha Srivastava
Amelia Hardy
John Thickstun
Esin Durmus
...
Hancheng Cao
Tony Lee
Rishi Bommasani
Michael S. Bernstein
Abigail Z. Jacobs
LM&MA
ALM
309
119
0
19 Dec 2022
ALERT: Adapting Language Models to Reasoning Tasks
Annual Meeting of the Association for Computational Linguistics (ACL), 2022
Ping Yu
Tianlu Wang
O. Yu. Golovneva
Badr AlKhamissi
Siddharth Verma
Zhijing Jin
Gargi Ghosh
Mona T. Diab
Asli Celikyilmaz
ReLM
LRM
269
20
0
16 Dec 2022
On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning
Annual Meeting of the Association for Computational Linguistics (ACL), 2022
Omar Shaikh
Hongxin Zhang
William B. Held
Michael S. Bernstein
Diyi Yang
ReLM
LRM
488
241
0
15 Dec 2022
Automaton-Based Representations of Task Knowledge from Generative Language Models
Yunhao Yang
Jean-Raphael Gaglione
Cyrus Neary
Ufuk Topcu
422
14
0
04 Dec 2022
SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
International Conference on Machine Learning (ICML), 2022
Guangxuan Xiao
Ji Lin
Mickael Seznec
Hao Wu
Julien Demouth
Song Han
MQ
807
1,219
0
18 Nov 2022
Galactica: A Large Language Model for Science
Ross Taylor
Marcin Kardas
Guillem Cucurull
Thomas Scialom
Anthony Hartshorn
Elvis Saravia
Andrew Poulton
Viktor Kerkez
Robert Stojnic
ELM
ReLM
396
937
0
16 Nov 2022
Calibrated Interpretation: Confidence Estimation in Semantic Parsing
Transactions of the Association for Computational Linguistics (TACL), 2022
Elias Stengel-Eskin
Benjamin Van Durme
UQLM
447
36
0
14 Nov 2022
Measuring Progress on Scalable Oversight for Large Language Models
Sam Bowman
Jeeyoon Hyun
Ethan Perez
Edwin Chen
Craig Pettit
...
Tristan Hume
Yuntao Bai
Zac Hatfield-Dodds
Benjamin Mann
Jared Kaplan
ALM
ELM
325
176
0
04 Nov 2022
LMentry: A Language Model Benchmark of Elementary Language Tasks
Annual Meeting of the Association for Computational Linguistics (ACL), 2022
Avia Efrat
Or Honovich
Omer Levy
239
31
0
03 Nov 2022
RQUGE: Reference-Free Metric for Evaluating Question Generation by Answering the Question
Annual Meeting of the Association for Computational Linguistics (ACL), 2022
Alireza Mohammadshahi
Thomas Scialom
Majid Yazdani
Pouya Yanki
Angela Fan
James Henderson
Marzieh Saeidi
267
24
0
02 Nov 2022
Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models
International Conference on Learning Representations (ICLR), 2022
Xiaoman Pan
Wenlin Yao
Hongming Zhang
Dian Yu
Dong Yu
Jianshu Chen
KELM
572
28
0
28 Oct 2022
Leveraging Large Language Models for Multiple Choice Question Answering
International Conference on Learning Representations (ICLR), 2022
Joshua Robinson
Christopher Rytting
David Wingate
ELM
410
244
0
22 Oct 2022
Scaling Instruction-Finetuned Language Models
Journal of machine learning research (JMLR), 2022
Hyung Won Chung
Le Hou
Shayne Longpre
Barret Zoph
Yi Tay
...
Jacob Devlin
Adam Roberts
Denny Zhou
Quoc V. Le
Jason W. Wei
ReLM
LRM
1.5K
3,822
0
20 Oct 2022
Transcending Scaling Laws with 0.1% Extra Compute
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022
Yi Tay
Jason W. Wei
Hyung Won Chung
Vinh Q. Tran
David R. So
...
Donald Metzler
Slav Petrov
N. Houlsby
Quoc V. Le
Mostafa Dehghani
LRM
314
73
0
20 Oct 2022
RARR: Researching and Revising What Language Models Say, Using Language Models
Annual Meeting of the Association for Computational Linguistics (ACL), 2022
Luyu Gao
Zhuyun Dai
Panupong Pasupat
Anthony Chen
Arun Tejasvi Chaganty
...
Vincent Zhao
Ni Lao
Hongrae Lee
Da-Cheng Juan
Kelvin Guu
HILM
KELM
712
282
0
17 Oct 2022
Mind's Eye: Grounded Language Model Reasoning through Simulation
International Conference on Learning Representations (ICLR), 2022
Ruibo Liu
Jason W. Wei
S. Gu
Te-Yen Wu
Soroush Vosoughi
Claire Cui
Denny Zhou
Andrew M. Dai
ReLM
LRM
362
95
0
11 Oct 2022
Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learners
International Conference on Learning Representations (ICLR), 2022
Seonghyeon Ye
Doyoung Kim
Joel Jang
Joongbo Shin
Minjoon Seo
FedML
VLM
UQCV
LRM
451
25
0
06 Oct 2022
GLM-130B: An Open Bilingual Pre-trained Model
International Conference on Learning Representations (ICLR), 2022
Aohan Zeng
Xiao Liu
Zhengxiao Du
Zihan Wang
Hanyu Lai
...
Jidong Zhai
Wenguang Chen
Peng Zhang
Yuxiao Dong
Jie Tang
BDL
LRM
805
1,221
0
05 Oct 2022
Improving alignment of dialogue agents via targeted human judgements
Amelia Glaese
Nat McAleese
Maja Trkebacz
John Aslanides
Vlad Firoiu
...
John F. J. Mellor
Demis Hassabis
Koray Kavukcuoglu
Lisa Anne Hendricks
G. Irving
ALM
AAML
538
637
0
28 Sep 2022
Variational Open-Domain Question Answering
International Conference on Machine Learning (ICML), 2022
Valentin Liévin
Andreas Geert Motzfeldt
Ida Riis Jensen
Ole Winther
OOD
BDL
213
11
0
23 Sep 2022
Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies
International Conference on Machine Learning (ICML), 2022
Gati Aher
RosaI. Arriaga
Adam Tauman Kalai
654
558
0
18 Aug 2022
Social Simulacra: Creating Populated Prototypes for Social Computing Systems
ACM Symposium on User Interface Software and Technology (UIST), 2022
Cristina Mata
Lindsay Popowski
Carrie J. Cai
Meredith Ringel Morris
Abigail Z. Jacobs
Michael S. Bernstein
306
393
0
08 Aug 2022
Can large language models reason about medical questions?
Patterns (Patterns), 2022
Valentin Liévin
C. Hother
Andreas Geert Motzfeldt
Ole Winther
ELM
LM&MA
AI4MH
LRM
518
397
0
17 Jul 2022
Language Models (Mostly) Know What They Know
Saurav Kadavath
Tom Conerly
Amanda Askell
T. Henighan
Dawn Drain
...
Nicholas Joseph
Benjamin Mann
Sam McCandlish
C. Olah
Jared Kaplan
ELM
638
1,181
0
11 Jul 2022
Forecasting Future World Events with Neural Networks
Neural Information Processing Systems (NeurIPS), 2022
Andy Zou
Tristan Xiao
Ryan Jia
Joe Kwon
Mantas Mazeika
Richard Li
Dawn Song
Jacob Steinhardt
Owain Evans
Dan Hendrycks
370
38
0
30 Jun 2022
Solving Quantitative Reasoning Problems with Language Models
Neural Information Processing Systems (NeurIPS), 2022
Aitor Lewkowycz
Anders Andreassen
David Dohan
Ethan Dyer
Henryk Michalewski
...
Theo Gutman-Solo
Yuhuai Wu
Behnam Neyshabur
Guy Gur-Ari
Vedant Misra
ReLM
ELM
LRM
664
1,322
0
29 Jun 2022
Emergent Abilities of Large Language Models
Jason W. Wei
Yi Tay
Rishi Bommasani
Colin Raffel
Barret Zoph
...
Tatsunori Hashimoto
Oriol Vinyals
Abigail Z. Jacobs
J. Dean
W. Fedus
ELM
ReLM
LRM
532
3,171
0
15 Jun 2022
From Human Days to Machine Seconds: Automatically Answering and Generating Machine Learning Final Exams
Knowledge Discovery and Data Mining (KDD), 2022
Iddo Drori
Sarah J. Zhang
Reece Shuttleworth
Sarah Zhang
Keith Tyser
...
Yann Hicke
Sage Simhon
S. Karnik
Darnell Granberry
Madeleine Udell
ELM
339
15
0
11 Jun 2022
A Survey in Mathematical Language Processing
Jordan Meadows
André Freitas
AIMat
244
18
0
30 May 2022
Previous
1
2
3
...
88
89
90
Next
Page 89 of 90
Page
of 90
Go
