Generalization through Memorization: Nearest Neighbor Language Models

1 November 2019

Urvashi Khandelwal

Omer Levy

Dan Jurafsky

Luke Zettlemoyer

M. Lewis

RALM

ArXiv PDF HTML

Papers citing "Generalization through Memorization: Nearest Neighbor Language Models"

26 / 576 papers shown

Title
Limitations of Autoregressive Models and Their Alternatives Chu-cheng Lin Aaron Jaech Xin Li Matthew R. Gormley Jason Eisner 29 58 0 22 Oct 2020
Explaining and Improving Model Behavior with k Nearest Neighbor Representations Nazneen Rajani Ben Krause Wengpeng Yin Tong Niu R. Socher Caiming Xiong FAtt 6 31 0 18 Oct 2020
Example-Driven Intent Prediction with Observers Shikib Mehri Mihail Eric 20 39 0 17 Oct 2020
Large Product Key Memory for Pretrained Language Models Gyuwan Kim Tae-Hwan Jung VLM KELM 10 2 0 08 Oct 2020
Learning to Recombine and Resample Data for Compositional Generalization Ekin Akyürek Afra Feyza Akyürek Jacob Andreas 16 79 0 08 Oct 2020
Efficient Meta Lifelong-Learning with Limited Memory Zirui Wang Sanket Vaibhav Mehta Barnabás Póczós J. Carbonell CLL KELM 14 74 0 06 Oct 2020
Nearest Neighbor Machine Translation Urvashi Khandelwal Angela Fan Dan Jurafsky Luke Zettlemoyer M. Lewis RALM 13 35 0 01 Oct 2020
Case-Based Abductive Natural Language Inference Marco Valentino Mokanarangan Thayaparan André Freitas 8 5 0 30 Sep 2020
Controllable Text Generation with Focused Variation Lei Shu Alexandros Papangelis Yi-Chia Wang Gökhan Tür Hu Xu Zhaleh Feizollahi Bing-Quan Liu Piero Molino 6 10 0 25 Sep 2020
Grounded Compositional Outputs for Adaptive Language Modeling Nikolaos Pappas Phoebe Mulcaire Noah A. Smith KELM 17 7 0 24 Sep 2020
Taking Notes on the Fly Helps BERT Pre-training Qiyu Wu Chen Xing Yatao Li Guolin Ke Di He Tie-Yan Liu 6 10 0 04 Aug 2020
Neural Language Generation: Formulation, Methods, and Evaluation Cristina Garbacea Qiaozhu Mei 37 30 0 31 Jul 2020
Neural Composition: Learning to Generate from Multiple Models Denis Filimonov R. Gadde Ariya Rastrow 8 3 0 10 Jul 2020
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval Lee Xiong Chenyan Xiong Ye Li Kwok-Fung Tang Jialin Liu Paul N. Bennett Junaid Ahmed Arnold Overwijk 11 1,177 0 01 Jul 2020
Learning Sparse Prototypes for Text Generation Junxian He Taylor Berg-Kirkpatrick Graham Neubig 11 23 0 29 Jun 2020
Train and You'll Miss It: Interactive Model Iteration with Weak Supervision and Pre-Trained Embeddings Mayee F. Chen Daniel Y. Fu Frederic Sala Sen Wu Ravi Teja Mullapudi Fait Poms Kayvon Fatahalian Christopher Ré 17 10 0 26 Jun 2020
Pre-training via Paraphrasing M. Lewis Marjan Ghazvininejad Gargi Ghosh Armen Aghajanyan Sida I. Wang Luke Zettlemoyer AIMat 17 158 0 26 Jun 2020
A Simple Approach to Case-Based Reasoning in Knowledge Bases Rajarshi Das Ameya Godbole S. Dhuliawala Manzil Zaheer Andrew McCallum 14 24 0 25 Jun 2020
Cross-lingual Retrieval for Iterative Self-Supervised Training C. Tran Y. Tang Xian Li Jiatao Gu RALM 28 72 0 16 Jun 2020
BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA Nora Kassner Hinrich Schütze RALM 16 68 0 02 May 2020
Augmenting Transformers with KNN-Based Composite Memory for Dialogue Angela Fan Claire Gardent Chloé Braud Antoine Bordes RALM 34 75 0 27 Apr 2020
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks Suchin Gururangan Ana Marasović Swabha Swayamdipta Kyle Lo Iz Beltagy Doug Downey Noah A. Smith VLM AI4CE CLL 11 2,349 0 23 Apr 2020
Exemplar VAE: Linking Generative Models, Nearest Neighbor Retrieval, and Data Augmentation Sajad Norouzi David J. Fleet Mohammad Norouzi VLM DRL 6 3 0 09 Apr 2020
REALM: Retrieval-Augmented Language Model Pre-Training Kelvin Guu Kenton Lee Zora Tung Panupong Pasupat Ming-Wei Chang RALM 13 1,987 0 10 Feb 2020
Improving Transformer Models by Reordering their Sublayers Ofir Press Noah A. Smith Omer Levy 11 87 0 10 Nov 2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,817 0 17 Sep 2019