The LAMBADA dataset: Word prediction requiring a broad discourse context

20 June 2016

Papers citing "The LAMBADA dataset: Word prediction requiring a broad discourse context"

50 / 185 papers shown

Title
VNHSGE: VietNamese High School Graduation Examination Dataset for Large Language Models Dao Xuan-Quy Le Ngoc-Bich Vo The-Duy Phan Xuan-Dung Ngo Bac-Bien Nguyen Van-Tien Nguyen Thi-My-Thanh Nguyen Hong-Phuoc 27 16 0 20 May 2023
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining Sang Michael Xie Hieu H. Pham Xuanyi Dong Nan Du Hanxiao Liu Yifeng Lu Percy Liang Quoc V. Le Tengyu Ma Adams Wei Yu MoMe MoE 58 180 0 17 May 2023
PaLM 2 Technical Report Rohan Anil Andrew M. Dai Orhan Firat Melvin Johnson Dmitry Lepikhin ... Ce Zheng Wei Zhou Denny Zhou Slav Petrov Yonghui Wu ReLM LRM 128 1,152 0 17 May 2023
CodeGen2: Lessons for Training LLMs on Programming and Natural Languages Erik Nijkamp A. Ghobadzadeh Caiming Xiong Silvio Savarese Yingbo Zhou 155 164 0 03 May 2023
Multimodal Grounding for Embodied AI via Augmented Reality Headsets for Natural Language Driven Task Planning Selma Wanna Fabian Parra R. Valner Karl Kruusamäe Mitch Pryor LM&Ro 28 2 0 26 Apr 2023
Training Large Language Models Efficiently with Sparsity and Dataflow V. Srinivasan Darshan Gandhi Urmish Thakker R. Prabhakar MoE 43 6 0 11 Apr 2023
RPTQ: Reorder-based Post-training Quantization for Large Language Models Zhihang Yuan Lin Niu Jia-Wen Liu Wenyu Liu Xinggang Wang Yuzhang Shang Guangyu Sun Qiang Wu Jiaxiang Wu Bingzhe Wu MQ 35 79 0 03 Apr 2023
Verbal behavior without syntactic structures: beyond Skinner and Chomsky S. Edelman 11 0 0 11 Mar 2023
CrossCodeBench: Benchmarking Cross-Task Generalization of Source Code Models Changan Niu Chuanyi Li Vincent Ng Bin Luo ELM ALM 43 9 0 08 Feb 2023
Optimus-CC: Efficient Large NLP Model Training with 3D Parallelism Aware Communication Compression Jaeyong Song Jinkyu Yim Jaewon Jung Hongsun Jang H. Kim Youngsok Kim Jinho Lee GNN 34 25 0 24 Jan 2023
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot Elias Frantar Dan Alistarh VLM 41 643 0 02 Jan 2023
Inconsistencies in Masked Language Models Tom Young Yunan Chen Yang You 24 2 0 30 Dec 2022
The case for 4-bit precision: k-bit Inference Scaling Laws Tim Dettmers Luke Zettlemoyer MQ 27 218 0 19 Dec 2022
DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing Conglong Li Z. Yao Xiaoxia Wu Minjia Zhang Connor Holmes Cheng Li Yuxiong He 32 25 0 07 Dec 2022
Validating Large Language Models with ReLM Michael Kuchnik Virginia Smith George Amvrosiadis 38 27 0 21 Nov 2022
SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models Guangxuan Xiao Ji Lin Mickael Seznec Hao Wu Julien Demouth Song Han MQ 101 749 0 18 Nov 2022
Random-LTD: Random and Layerwise Token Dropping Brings Efficient Training for Large-scale Transformers Z. Yao Xiaoxia Wu Conglong Li Connor Holmes Minjia Zhang Cheng-rong Li Yuxiong He 31 11 0 17 Nov 2022
GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers Elias Frantar Saleh Ashkboos Torsten Hoefler Dan Alistarh MQ 33 905 0 31 Oct 2022
Towards Better Few-Shot and Finetuning Performance with Forgetful Causal Language Models Hao Liu Xinyang Geng Lisa Lee Igor Mordatch Sergey Levine Sharan Narang Pieter Abbeel KELM CLL 35 2 0 24 Oct 2022
The Better Your Syntax, the Better Your Semantics? Probing Pretrained Language Models for the English Comparative Correlative Leonie Weissweiler Valentin Hofmann Abdullatif Köksal Hinrich Schütze 40 33 0 24 Oct 2022
Multitasking Models are Robust to Structural Failure: A Neural Model for Bilingual Cognitive Reserve Giannis Daras Negin Raoof Zoi Gkalitsiou A. Dimakis 35 2 0 20 Oct 2022
Scaling Instruction-Finetuned Language Models Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay ... Jacob Devlin Adam Roberts Denny Zhou Quoc V. Le Jason W. Wei ReLM LRM 103 3,019 0 20 Oct 2022
Transcending Scaling Laws with 0.1% Extra Compute Yi Tay Jason W. Wei Hyung Won Chung Vinh Q. Tran David R. So ... Donald Metzler Slav Petrov N. Houlsby Quoc V. Le Mostafa Dehghani LRM 47 68 0 20 Oct 2022
Small Character Models Match Large Word Models for Autocomplete Under Memory Constraints Ganesh Jawahar Subhabrata Mukherjee Debadeepta Dey Muhammad Abdul-Mageed L. Lakshmanan C. C. T. Mendes Gustavo de Rosa S. Shah 35 0 0 06 Oct 2022
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng Zhang Yuxiao Dong Jie Tang BDL LRM 275 1,077 0 05 Oct 2022
Language Models (Mostly) Know What They Know Saurav Kadavath Tom Conerly Amanda Askell T. Henighan Dawn Drain ... Nicholas Joseph Benjamin Mann Sam McCandlish C. Olah Jared Kaplan ELM 61 726 0 11 Jul 2022
ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers Z. Yao Reza Yazdani Aminabadi Minjia Zhang Xiaoxia Wu Conglong Li Yuxiong He VLM MQ 73 448 0 04 Jun 2022
Visually-Augmented Language Modeling Weizhi Wang Li Dong Hao Cheng Haoyu Song Xiaodong Liu Xifeng Yan Jianfeng Gao Furu Wei VLM 38 18 0 20 May 2022
When a sentence does not introduce a discourse entity, Transformer-based models still sometimes refer to it Sebastian Schuster Tal Linzen 18 25 0 06 May 2022
Entity Cloze By Date: What LMs Know About Unseen Entities Yasumasa Onoe Michael J.Q. Zhang Eunsol Choi Greg Durrett KELM 29 49 0 05 May 2022
Improving In-Context Few-Shot Learning via Self-Supervised Training Mingda Chen Jingfei Du Ramakanth Pasunuru Todor Mihaylov Srini Iyer Ves Stoyanov Zornitsa Kozareva SSL AI4MH 47 64 0 03 May 2022
A Survey of Machine Narrative Reading Comprehension Assessments Yisi Sang Xiangyang Mou Jing Li Jeffrey Stanton Mo Yu 13 14 0 30 Apr 2022
GPT-NeoX-20B: An Open-Source Autoregressive Language Model Sid Black Stella Biderman Eric Hallahan Quentin G. Anthony Leo Gao ... Shivanshu Purohit Laria Reynolds J. Tow Benqi Wang Samuel Weinbach 102 803 0 14 Apr 2022
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback Yuntao Bai Andy Jones Kamal Ndousse Amanda Askell Anna Chen ... Jack Clark Sam McCandlish C. Olah Benjamin Mann Jared Kaplan 107 2,352 0 12 Apr 2022
What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization? Thomas Wang Adam Roberts Daniel Hesslow Teven Le Scao Hyung Won Chung Iz Beltagy Julien Launay Colin Raffel 50 168 0 12 Apr 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 168 6,035 0 05 Apr 2022
Evaluating Prompts Across Multiple Choice Tasks In a Zero-Shot Setting Gabriel Orlanski LRM 29 2 0 29 Mar 2022
Training Compute-Optimal Large Language Models Jordan Hoffmann Sebastian Borgeaud A. Mensch Elena Buchatskaya Trevor Cai ... Karen Simonyan Erich Elsen Jack W. Rae Oriol Vinyals Laurent Sifre AI4TS 69 1,856 0 29 Mar 2022
CoDA21: Evaluating Language Understanding Capabilities of NLP Models With Context-Definition Alignment Lutfi Kerem Senel Timo Schick Hinrich Schütze ELM ALM 31 5 0 11 Mar 2022
Improving Lexical Embeddings for Robust Question Answering Weiwen Xu Bowei Zou Wai Lam Ai Ti Aw OOD AAML 21 1 0 28 Feb 2022
Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models Wei Ping Ming-Yu Liu Chaowei Xiao Peng Xu M. Patwary Mohammad Shoeybi Bo Li Anima Anandkumar Bryan Catanzaro 31 65 0 08 Feb 2022
Unified Scaling Laws for Routed Language Models Aidan Clark Diego de Las Casas Aurelia Guy A. Mensch Michela Paganini ... Oriol Vinyals Jack W. Rae Erich Elsen Koray Kavukcuoglu Karen Simonyan MoE 27 177 0 02 Feb 2022
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model Shaden Smith M. Patwary Brandon Norick P. LeGresley Samyam Rajbhandari ... Mohammad Shoeybi Yuxiong He Michael Houston Saurabh Tiwary Bryan Catanzaro MoE 95 733 0 28 Jan 2022
DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale Samyam Rajbhandari Conglong Li Z. Yao Minjia Zhang Reza Yazdani Aminabadi A. A. Awan Jeff Rasley Yuxiong He 47 288 0 14 Jan 2022
Multi Document Reading Comprehension Avi Chawla 30 0 0 05 Jan 2022
Native Chinese Reader: A Dataset Towards Native-Level Chinese Machine Reading Comprehension Shusheng Xu Yichen Liu Xiaoyuan Yi Siyuan Zhou Huizi Li Yi Wu ELM 31 3 0 13 Dec 2021
Improving language models by retrieving from trillions of tokens Sebastian Borgeaud A. Mensch Jordan Hoffmann Trevor Cai Eliza Rutherford ... Simon Osindero Karen Simonyan Jack W. Rae Erich Elsen Laurent Sifre KELM RALM 90 1,031 0 08 Dec 2021
An Explanation of In-context Learning as Implicit Bayesian Inference Sang Michael Xie Aditi Raghunathan Percy Liang Tengyu Ma ReLM BDL VPVLM LRM 93 698 0 03 Nov 2021
Colossal-AI: A Unified Deep Learning System For Large-Scale Parallel Training Yongbin Li Hongxin Liu Zhengda Bian Boxiang Wang Haichen Huang Fan Cui Chuan-Qing Wang Yang You GNN 30 143 0 28 Oct 2021
Coherence boosting: When your pretrained language model is not paying enough attention Nikolay Malkin Zhen Wang Nebojsa Jojic RALM 21 35 0 15 Oct 2021