v1v2v3 (latest)

Measuring Massive Multitask Language Understanding

International Conference on Learning Representations (ICLR), 2020

7 September 2020

ArXiv (abs)PDF HTML HuggingFace (3 upvotes)

Papers citing "Measuring Massive Multitask Language Understanding"

50 / 4,428 papers shown

Title
Can GPT-3 Perform Statutory Reasoning?International Conference on Artificial Intelligence and Law (ICAIL), 2023 Andrew Blair-Stanek Nils Holzenberger Benjamin Van Durme ELM LRM 259 113 0 13 Feb 2023
Mathematical Capabilities of ChatGPTNeural Information Processing Systems (NeurIPS), 2023 Simon Frieder Luca Pinchetti Alexis Chevalier Ryan-Rhys Griffiths Tommaso Salvatori Thomas Lukasiewicz P. Petersen Julius Berner ELM AI4MH 372 512 0 31 Jan 2023
The Flan Collection: Designing Data and Methods for Effective Instruction TuningInternational Conference on Machine Learning (ICML), 2023 Shayne Longpre Le Hou Tu Vu Albert Webson Hyung Won Chung ... Denny Zhou Quoc V. Le Barret Zoph Jason W. Wei Adam Roberts ALM 340 826 0 31 Jan 2023
LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal DomainConference on Empirical Methods in Natural Language Processing (EMNLP), 2023 Joel Niklaus Veton Matoshi Pooja Rani Andrea Galassi Matthias Sturmer Ilias Chalkidis ELM AILaw 303 72 0 30 Jan 2023
REPLUG: Retrieval-Augmented Black-Box Language ModelsNorth American Chapter of the Association for Computational Linguistics (NAACL), 2023 Weijia Shi Sewon Min Michihiro Yasunaga Minjoon Seo Rich James M. Lewis Luke Zettlemoyer Anuj Kumar RALM VLM KELM 559 816 0 30 Jan 2023
ThoughtSource: A central hub for large language model reasoning dataScientific Data (Sci Data), 2023 Simon Ott Konstantin Hebenstreit Valentin Liévin C. Hother M. Moradi Maximilian Mayrhauser Robert Praas Ole Winther Matthias Samwald ReLM LRM 390 57 0 27 Jan 2023
MAUD: An Expert-Annotated Legal NLP Dataset for Merger Agreement UnderstandingConference on Empirical Methods in Natural Language Processing (EMNLP), 2023 Steven H. Wang Antoine Scardigli Leonard Tang Wei Chen D.M. Levkin Anya Chen Spencer Ball Thomas Woodside Oliver Zhang Dan Hendrycks AILaw ELM 172 34 0 02 Jan 2023
Inconsistencies in Masked Language Models Tom Young Yunan Chen Yang You 233 2 0 30 Dec 2022
Large Language Models Encode Clinical KnowledgeNature (Nature), 2022 K. Singhal Shekoofeh Azizi T. Tu S. S. Mahdavi Jason W. Wei ... A. Rajkomar Joelle Barral Christopher Semturs Alan Karthikesalingam Vivek Natarajan LM&MA ELM AI4MH 546 3,223 0 26 Dec 2022
Quality at the Tail of Machine Learning Inference Zhengxin Yang Wanling Gao Chunjie Luo Lei Wang Fei Tang Xu Wen Jianfeng Zhan 160 1 0 25 Dec 2022
OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization Srinivasan Iyer Xi Lin Ramakanth Pasunuru Todor Mihaylov Daniel Simig ... Jeff Wang Christopher Dewan Asli Celikyilmaz Luke Zettlemoyer Veselin Stoyanov ALM 359 300 0 22 Dec 2022
ORCA: A Challenging Benchmark for Arabic Language UnderstandingAnnual Meeting of the Association for Computational Linguistics (ACL), 2022 AbdelRahim Elmadany El Moatez Billah Nagoudi Muhammad Abdul-Mageed ELM 229 59 0 21 Dec 2022
A Survey of Deep Learning for Mathematical ReasoningAnnual Meeting of the Association for Computational Linguistics (ACL), 2022 Pan Lu Liang Qiu Wenhao Yu Sean Welleck Kai-Wei Chang ReLM LRM 211 169 0 20 Dec 2022
Evaluating Human-Language Model Interaction Mina Lee Megha Srivastava Amelia Hardy John Thickstun Esin Durmus ... Hancheng Cao Tony Lee Rishi Bommasani Michael S. Bernstein Abigail Z. Jacobs LM&MA ALM 254 117 0 19 Dec 2022
ALERT: Adapting Language Models to Reasoning TasksAnnual Meeting of the Association for Computational Linguistics (ACL), 2022 Ping Yu Tianlu Wang O. Yu. Golovneva Badr AlKhamissi Siddharth Verma Zhijing Jin Gargi Ghosh Mona T. Diab Asli Celikyilmaz ReLM LRM 193 20 0 16 Dec 2022
On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot ReasoningAnnual Meeting of the Association for Computational Linguistics (ACL), 2022 Omar Shaikh Hongxin Zhang William B. Held Michael S. Bernstein Diyi Yang ReLM LRM 395 233 0 15 Dec 2022
Automaton-Based Representations of Task Knowledge from Generative Language Models Yunhao Yang Jean-Raphael Gaglione Cyrus Neary Ufuk Topcu 370 14 0 04 Dec 2022
SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language ModelsInternational Conference on Machine Learning (ICML), 2022 Guangxuan Xiao Ji Lin Mickael Seznec Hao Wu Julien Demouth Song Han MQ 637 1,148 0 18 Nov 2022
Galactica: A Large Language Model for Science Ross Taylor Marcin Kardas Guillem Cucurull Thomas Scialom Anthony Hartshorn Elvis Saravia Andrew Poulton Viktor Kerkez Robert Stojnic ELM ReLM 295 907 0 16 Nov 2022
Calibrated Interpretation: Confidence Estimation in Semantic ParsingTransactions of the Association for Computational Linguistics (TACL), 2022 Elias Stengel-Eskin Benjamin Van Durme UQLM 298 33 0 14 Nov 2022
Measuring Progress on Scalable Oversight for Large Language Models Sam Bowman Jeeyoon Hyun Ethan Perez Edwin Chen Craig Pettit ... Tristan Hume Yuntao Bai Zac Hatfield-Dodds Benjamin Mann Jared Kaplan ALM ELM 234 166 0 04 Nov 2022
LMentry: A Language Model Benchmark of Elementary Language TasksAnnual Meeting of the Association for Computational Linguistics (ACL), 2022 Avia Efrat Or Honovich Omer Levy 187 29 0 03 Nov 2022
RQUGE: Reference-Free Metric for Evaluating Question Generation by Answering the QuestionAnnual Meeting of the Association for Computational Linguistics (ACL), 2022 Alireza Mohammadshahi Thomas Scialom Majid Yazdani Pouya Yanki Angela Fan James Henderson Marzieh Saeidi 219 21 0 02 Nov 2022
Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language ModelsInternational Conference on Learning Representations (ICLR), 2022 Xiaoman Pan Wenlin Yao Hongming Zhang Dian Yu Dong Yu Jianshu Chen KELM 447 26 0 28 Oct 2022
Leveraging Large Language Models for Multiple Choice Question AnsweringInternational Conference on Learning Representations (ICLR), 2022 Joshua Robinson Christopher Rytting David Wingate ELM 368 236 0 22 Oct 2022
Scaling Instruction-Finetuned Language ModelsJournal of machine learning research (JMLR), 2022 Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay ... Jacob Devlin Adam Roberts Denny Zhou Quoc V. Le Jason W. Wei ReLM LRM 864 3,738 0 20 Oct 2022
Transcending Scaling Laws with 0.1% Extra ComputeConference on Empirical Methods in Natural Language Processing (EMNLP), 2022 Yi Tay Jason W. Wei Hyung Won Chung Vinh Q. Tran David R. So ... Donald Metzler Slav Petrov N. Houlsby Quoc V. Le Mostafa Dehghani LRM 237 73 0 20 Oct 2022
RARR: Researching and Revising What Language Models Say, Using Language ModelsAnnual Meeting of the Association for Computational Linguistics (ACL), 2022 Luyu Gao Zhuyun Dai Panupong Pasupat Anthony Chen Arun Tejasvi Chaganty ... Vincent Zhao Ni Lao Hongrae Lee Da-Cheng Juan Kelvin Guu HILM KELM 574 277 0 17 Oct 2022
Mind's Eye: Grounded Language Model Reasoning through SimulationInternational Conference on Learning Representations (ICLR), 2022 Ruibo Liu Jason W. Wei S. Gu Te-Yen Wu Soroush Vosoughi Claire Cui Denny Zhou Andrew M. Dai ReLM LRM 308 91 0 11 Oct 2022
Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot LearnersInternational Conference on Learning Representations (ICLR), 2022 Seonghyeon Ye Doyoung Kim Joel Jang Joongbo Shin Minjoon Seo FedML VLM UQCV LRM 359 25 0 06 Oct 2022
GLM-130B: An Open Bilingual Pre-trained ModelInternational Conference on Learning Representations (ICLR), 2022 Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng Zhang Yuxiao Dong Jie Tang BDL LRM 607 1,204 0 05 Oct 2022
Improving alignment of dialogue agents via targeted human judgements Amelia Glaese Nat McAleese Maja Trkebacz John Aslanides Vlad Firoiu ... John F. J. Mellor Demis Hassabis Koray Kavukcuoglu Lisa Anne Hendricks G. Irving ALM AAML 473 622 0 28 Sep 2022
Variational Open-Domain Question AnsweringInternational Conference on Machine Learning (ICML), 2022 Valentin Liévin Andreas Geert Motzfeldt Ida Riis Jensen Ole Winther OOD BDL 166 11 0 23 Sep 2022
Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject StudiesInternational Conference on Machine Learning (ICML), 2022 Gati Aher RosaI. Arriaga Adam Tauman Kalai 503 530 0 18 Aug 2022
Social Simulacra: Creating Populated Prototypes for Social Computing SystemsACM Symposium on User Interface Software and Technology (UIST), 2022 Cristina Mata Lindsay Popowski Carrie J. Cai Meredith Ringel Morris Abigail Z. Jacobs Michael S. Bernstein 228 375 0 08 Aug 2022
Can large language models reason about medical questions?Patterns (Patterns), 2022 Valentin Liévin C. Hother Andreas Geert Motzfeldt Ole Winther ELM LM&MA AI4MH LRM 442 379 0 17 Jul 2022
Language Models (Mostly) Know What They Know Saurav Kadavath Tom Conerly Amanda Askell T. Henighan Dawn Drain ... Nicholas Joseph Benjamin Mann Sam McCandlish C. Olah Jared Kaplan ELM 474 1,094 0 11 Jul 2022
Forecasting Future World Events with Neural NetworksNeural Information Processing Systems (NeurIPS), 2022 Andy Zou Tristan Xiao Ryan Jia Joe Kwon Mantas Mazeika Richard Li Dawn Song Jacob Steinhardt Owain Evans Dan Hendrycks 272 35 0 30 Jun 2022
Solving Quantitative Reasoning Problems with Language ModelsNeural Information Processing Systems (NeurIPS), 2022 Aitor Lewkowycz Anders Andreassen David Dohan Ethan Dyer Henryk Michalewski ... Theo Gutman-Solo Yuhuai Wu Behnam Neyshabur Guy Gur-Ari Vedant Misra ReLM ELM LRM 588 1,249 0 29 Jun 2022
Emergent Abilities of Large Language Models Jason W. Wei Yi Tay Rishi Bommasani Colin Raffel Barret Zoph ... Tatsunori Hashimoto Oriol Vinyals Abigail Z. Jacobs J. Dean W. Fedus ELM ReLM LRM 440 3,046 0 15 Jun 2022
From Human Days to Machine Seconds: Automatically Answering and Generating Machine Learning Final ExamsKnowledge Discovery and Data Mining (KDD), 2022 Iddo Drori Sarah J. Zhang Reece Shuttleworth Sarah Zhang Keith Tyser ... Yann Hicke Sage Simhon S. Karnik Darnell Granberry Madeleine Udell ELM 219 13 0 11 Jun 2022
A Survey in Mathematical Language Processing Jordan Meadows André Freitas AIMat 156 18 0 30 May 2022
Teaching Models to Express Their Uncertainty in Words Stephanie C. Lin Jacob Hilton Owain Evans OOD 415 523 0 28 May 2022
Eliciting and Understanding Cross-Task Skills with Task-Level Mixture-of-ExpertsConference on Empirical Methods in Natural Language Processing (EMNLP), 2022 Qinyuan Ye Juan Zha Xiang Ren MoE 211 16 0 25 May 2022
UL2: Unifying Language Learning ParadigmsInternational Conference on Learning Representations (ICLR), 2022 Yi Tay Mostafa Dehghani Vinh Q. Tran Xavier Garcia Jason W. Wei ... Tal Schuster H. Zheng Denny Zhou N. Houlsby Donald Metzler AI4CE 429 354 0 10 May 2022
GPT-NeoX-20B: An Open-Source Autoregressive Language Model Sid Black Stella Biderman Eric Hallahan Quentin G. Anthony Leo Gao ... Shivanshu Purohit Laria Reynolds J. Tow Benqi Wang Samuel Weinbach 302 930 0 14 Apr 2022
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback Yuntao Bai Andy Jones Kamal Ndousse Amanda Askell Anna Chen ... Jack Clark Sam McCandlish C. Olah Benjamin Mann Jared Kaplan 737 3,383 0 12 Apr 2022
NumGLUE: A Suite of Fundamental yet Challenging Mathematical Reasoning TasksAnnual Meeting of the Association for Computational Linguistics (ACL), 2022 Swaroop Mishra Arindam Mitra Neeraj Varshney Bhavdeep Singh Sachdeva Peter Clark Chitta Baral Ashwin Kalyan AIMat ReLM ELM LRM 204 120 0 12 Apr 2022
PaLM: Scaling Language Modeling with PathwaysJournal of machine learning research (JMLR), 2022 Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 1.1K 7,275 0 05 Apr 2022
LinkBERT: Pretraining Language Models with Document LinksAnnual Meeting of the Association for Computational Linguistics (ACL), 2022 Michihiro Yasunaga J. Leskovec Abigail Z. Jacobs KELM 194 419 0 29 Mar 2022