Generalization through Memorization: Nearest Neighbor Language Models

1 November 2019

Urvashi Khandelwal

Omer Levy

Dan Jurafsky

Luke Zettlemoyer

M. Lewis

RALM

ArXiv PDF HTML

Papers citing "Generalization through Memorization: Nearest Neighbor Language Models"

50 / 576 papers shown

Title
Nearest Neighbour Few-Shot Learning for Cross-lingual Classification M Saiful Bari Batool Haider Saab Mansour VLM 11 13 0 06 Sep 2021
Combining Transformers with Natural Language Explanations Federico Ruggeri Marco Lippi Paolo Torroni 17 1 0 02 Sep 2021
$$\infty$-former: Infinite Memory Transformer$ $\infty$ -former: Infinite Memory Transformer Pedro Henrique Martins Zita Marinho André F. T. Martins 28 11 0 01 Sep 2021
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 245 695 0 27 Aug 2021
Towards Continual Entity Learning in Language Models for Conversational Agents R. Gadde I. Bulyko KELM 6 1 0 30 Jul 2021
Pointer Value Retrieval: A new benchmark for understanding the limits of neural network generalization Chiyuan Zhang M. Raghu Jon M. Kleinberg Samy Bengio OOD 19 30 0 27 Jul 2021
Internet-Augmented Dialogue Generation M. Komeili Kurt Shuster Jason Weston RALM 233 280 0 15 Jul 2021
On Training Instance Selection for Few-Shot Neural Text Generation Ernie Chang Xiaoyu Shen Hui-Syuan Yeh Vera Demberg 14 40 0 07 Jul 2021
Ascent Similarity Caching with Approximate Indexes T. Si Salem Giovanni Neglia D. Carra 12 7 0 02 Jul 2021
Memorization and Generalization in Neural Code Intelligence Models Md Rafiqul Islam Rabin Aftab Hussain Mohammad Amin Alipour Vincent J. Hellendoorn TDI 27 40 0 16 Jun 2021
End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answering Devendra Singh Sachan Siva Reddy William L. Hamilton Chris Dyer Dani Yogatama OOD RALM 21 160 0 09 Jun 2021
Improving Automated Evaluation of Open Domain Dialog via Diverse Reference Augmentation Varun Gangal Harsh Jhamtani Eduard H. Hovy Taylor Berg-Kirkpatrick 6 8 0 05 Jun 2021
Ember: No-Code Context Enrichment via Similarity-Based Keyless Joins S. Suri Ihab F. Ilyas Christopher Ré Theodoros Rekatsinas 25 21 0 02 Jun 2021
MOLEMAN: Mention-Only Linking of Entities with a Mention Annotation Network Nicholas FitzGerald Jan A. Botha D. Gillick Daniel M. Bikel Tom Kwiatkowski Andrew McCallum 24 15 0 02 Jun 2021
Fast Nearest Neighbor Machine Translation Yuxian Meng Xiaoya Li Xiayu Zheng Fei Wu Xiaofei Sun Tianwei Zhang Jiwei Li LRM 16 49 0 30 May 2021
Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation Zhiyong Wu Lingpeng Kong W. Bi Xiang Li B. Kao LRM 15 76 0 30 May 2021
Towards mental time travel: a hierarchical memory for reinforcement learning agents Andrew Kyle Lampinen Stephanie C. Y. Chan Andrea Banino Felix Hill 11 47 0 28 May 2021
Not Far Away, Not So Close: Sample Efficient Nearest Neighbour Data Augmentation via MiniMax Ehsan Kamalloo Mehdi Rezagholizadeh Peyman Passban Ali Ghodsi AAML 12 17 0 28 May 2021
Adaptive Nearest Neighbor Machine Translation Xin Zheng Zhirui Zhang Junliang Guo Shujian Huang Boxing Chen Weihua Luo Jiajun Chen 12 94 0 27 May 2021
Neural Machine Translation with Monolingual Translation Memory Deng Cai Yan Wang Huayang Li Wai Lam Lemao Liu 8 101 0 24 May 2021
Retrieval-Augmented Transformer-XL for Close-Domain Dialog Generation Giovanni Bonetta R. Cancelliere Ding Liu Paul Vozila RALM 14 16 0 19 May 2021
RetGen: A Joint framework for Retrieval and Grounded Text Generation Modeling Yizhe Zhang Siqi Sun Xiang Gao Yuwei Fang Chris Brockett Michel Galley Jianfeng Gao Bill Dolan RALM 22 30 0 14 May 2021
Paraphrastic Representations at Scale John Wieting Kevin Gimpel Graham Neubig Taylor Berg-Kirkpatrick 11 18 0 30 Apr 2021
Case-based Reasoning for Natural Language Queries over Knowledge Bases Rajarshi Das Manzil Zaheer Dung Ngoc Thai Ameya Godbole Ethan Perez Jay Yoon Lee Lizhen Tan L. Polymenakos Andrew McCallum 15 162 0 18 Apr 2021
Go Forth and Prosper: Language Modeling with Ancient Textual History Rik Koncel-Kedziorski Noah A. Smith KELM 6 0 0 18 Apr 2021
Generating Related Work Darsh J. Shah Regina Barzilay 26 3 0 18 Apr 2021
Cross-Modal Retrieval Augmentation for Multi-Modal Classification Shir Gur Natalia Neverova C. Stauffer Ser-Nam Lim Douwe Kiela A. Reiter 6 26 0 16 Apr 2021
Retrieval Augmentation Reduces Hallucination in Conversation Kurt Shuster Spencer Poff Moya Chen Douwe Kiela Jason Weston HILM 42 682 0 15 Apr 2021
Few-shot Intent Classification and Slot Filling with Retrieved Examples Dian Yu Luheng He Yuan Zhang Xinya Du Panupong Pasupat Qi Li VLM 15 50 0 12 Apr 2021
Lookup-Table Recurrent Language Models for Long Tail Speech Recognition W. R. Huang Tara N. Sainath Cal Peyser Shankar Kumar David Rybach Trevor Strohman RALM LMTD 14 5 0 09 Apr 2021
Revisiting Simple Neural Probabilistic Language Models Simeng Sun Mohit Iyyer 10 14 0 08 Apr 2021
Perspective, Survey and Trends: Public Driving Datasets and Toolsets for Autonomous Driving Virtual Test Pengliang Ji Li Ruan Yunzhi Xue Limin Xiao Qian Dong 20 8 0 01 Apr 2021
A Neighbourhood Framework for Resource-Lean Content Flagging Sheikh Muhammad Sarwar Dimitrina Zlatkova Momchil Hardalov Yoan Dinkov Isabelle Augenstein Preslav Nakov 11 5 0 31 Mar 2021
BASE Layers: Simplifying Training of Large, Sparse Models M. Lewis Shruti Bhosale Tim Dettmers Naman Goyal Luke Zettlemoyer MoE 25 273 0 30 Mar 2021
Structure Inducing Pre-Training Matthew B. A. McDermott Brendan Yap Peter Szolovits Marinka Zitnik 30 18 0 18 Mar 2021
Retrieval Augmentation for Deep Neural Networks R. Ramos Patrícia Pereira Helena Moniz Joao Paulo Carvalho Bruno Martins VLM 19 0 0 25 Feb 2021
When Attention Meets Fast Recurrence: Training Language Models with Reduced Compute Tao Lei RALM VLM 45 47 0 24 Feb 2021
Leveraging Reinforcement Learning for evaluating Robustness of KNN Search Algorithms Pramod Vadiraja Christoph Balada OOD 11 1 0 10 Feb 2021
Adaptive Semiparametric Language Models Dani Yogatama Cyprien de Masson dÁutume Lingpeng Kong KELM RALM 27 97 0 04 Feb 2021
Mind the Gap: Assessing Temporal Generalization in Neural Language Models Angeliki Lazaridou A. Kuncoro E. Gribovskaya Devang Agrawal Adam Liska ... Sebastian Ruder Dani Yogatama Kris Cao Susannah Young Phil Blunsom VLM 24 207 0 03 Feb 2021
CNN with large memory layers R. Karimov Yury Malkov Karim Iskakov Victor Lempitsky 19 0 0 27 Jan 2021
Data-to-text Generation by Splicing Together Nearest Neighbors Sam Wiseman A. Backurs K. Stratos 14 9 0 20 Jan 2021
Diagnostic Captioning: A Survey John Pavlopoulos Vasiliki Kougia Ion Androutsopoulos D. Papamichail 3DV MedIm 89 26 0 18 Jan 2021
What Makes Good In-Context Examples for GPT- $3$ ? Jiachang Liu Dinghan Shen Yizhe Zhang Bill Dolan Lawrence Carin Weizhu Chen AAML RALM 275 1,312 0 17 Jan 2021
Subformer: Exploring Weight Sharing for Parameter Efficiency in Generative Transformers Machel Reid Edison Marrese-Taylor Y. Matsuo MoE 6 48 0 01 Jan 2021
Shortformer: Better Language Modeling using Shorter Inputs Ofir Press Noah A. Smith M. Lewis 219 88 0 31 Dec 2020
FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debugging Han Guo Nazneen Rajani Peter Hase Mohit Bansal Caiming Xiong TDI 17 102 0 31 Dec 2020
Modifying Memories in Transformer Models Chen Zhu A. S. Rawat Manzil Zaheer Srinadh Bhojanapalli Daliang Li Felix X. Yu Sanjiv Kumar KELM 6 189 0 01 Dec 2020
Cross-Domain Generalization Through Memorization: A Study of Nearest Neighbors in Neural Duplicate Question Detection Yadollah Yaghoobzadeh Alexandre Rochette Timothy J. Hazen OOD 12 1 0 22 Nov 2020
Language Models are Open Knowledge Graphs Chenguang Wang Xiao Liu D. Song SSL KELM 21 135 0 22 Oct 2020