Generalization through Memorization: Nearest Neighbor Language Models

1 November 2019

Urvashi Khandelwal

Omer Levy

Dan Jurafsky

Luke Zettlemoyer

M. Lewis

RALM

ArXiv PDF HTML

Papers citing "Generalization through Memorization: Nearest Neighbor Language Models"

50 / 576 papers shown

Title
Can deep learning match the efficiency of human visual long-term memory in storing object details? Emin Orhan VLM OCL 20 0 0 27 Apr 2022
Semi-Parametric Neural Image Synthesis A. Blattmann Robin Rombach Kaan Oktay Jonas Muller Bjorn Ommer DiffM 28 28 0 25 Apr 2022
METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals Payal Bajaj Chenyan Xiong Guolin Ke Xiaodong Liu Di He Saurabh Tiwary Tie-Yan Liu Paul N. Bennett Xia Song Jianfeng Gao 42 32 0 13 Apr 2022
Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering Wenhu Chen Pat Verga Michiel de Jong John Wieting William W. Cohen RALM KELM 22 26 0 10 Apr 2022
Knowledge Infused Decoding Ruibo Liu Guoqing Zheng Shashank Gupta Radhika Gaonkar Chongyang Gao Soroush Vosoughi Milad Shokouhi Ahmed Hassan Awadallah KELM 25 14 0 06 Apr 2022
Knowledge Base Index Compression via Dimensionality and Precision Reduction Vilém Zouhar Marius Mosbach Miaoran Zhang Dietrich Klakow 16 2 0 06 Apr 2022
$k$ NN-NER: Named Entity Recognition with Nearest Neighbor Search Shuhe Wang Xiaoya Li Yuxian Meng Tianwei Zhang Rongbin Ouyang Jiwei Li Guoyin Wang 21 11 0 31 Mar 2022
Shoring Up the Foundations: Fusing Model Embeddings and Weak Supervision Mayee F. Chen Daniel Y. Fu Dyah Adila Michael Zhang Frederic Sala Kayvon Fatahalian Christopher Ré 24 20 0 24 Mar 2022
Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completion Kurt Shuster M. Komeili Leonard Adolphs Stephen Roller Arthur Szlam Jason Weston KELM 31 120 0 24 Mar 2022
An Empirical Study of Memorization in NLP Xiaosen Zheng Jing Jiang TDI 17 24 1 23 Mar 2022
Better Language Model with Hypernym Class Prediction Richard He Bai Tong Wang Alessandro Sordoni Peng Shi 76 15 0 21 Mar 2022
Memorizing Transformers Yuhuai Wu M. Rabe DeLesley S. Hutchins Christian Szegedy RALM 16 171 0 16 Mar 2022
Interpretability for Language Learners Using Example-Based Grammatical Error Correction Masahiro Kaneko Sho Takase Ayana Niwa Naoaki Okazaki 31 26 0 14 Mar 2022
Internet-augmented language models through few-shot prompting for open-domain question answering Angeliki Lazaridou E. Gribovskaya Wojciech Stokowiec N. Grigorev KELM LRM 18 131 0 10 Mar 2022
Retrieval Augmented Classification for Long-Tail Visual Recognition Alex Long Wei Yin Thalaiyasingam Ajanthan Vu-Linh Nguyen Pulak Purkait Ravi Garg Alan Blair Chunhua Shen A. Hengel 11 107 0 22 Feb 2022
Knowledge Base Question Answering by Case-based Reasoning over Subgraphs Rajarshi Das Ameya Godbole Ankita Rajaram Naik Elliot Tower Robin Jia Manzil Zaheer Hannaneh Hajishirzi Andrew McCallum 17 53 0 22 Feb 2022
What Does it Mean for a Language Model to Preserve Privacy? Hannah Brown Katherine Lee Fatemehsadat Mireshghallah Reza Shokri Florian Tramèr PILM 16 230 0 11 Feb 2022
A Survey on Retrieval-Augmented Text Generation Huayang Li Yixuan Su Deng Cai Yan Wang Lemao Liu RALM 45 197 0 02 Feb 2022
Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval Uri Alon Frank F. Xu Junxian He Sudipta Sengupta Dan Roth Graham Neubig RALM 70 62 0 28 Jan 2022
Relational Memory Augmented Language Models Qi Liu Dani Yogatama Phil Blunsom KELM RALM 61 32 0 24 Jan 2022
Artefact Retrieval: Overview of NLP Models with Knowledge Base Access Vilém Zouhar Marius Mosbach Debanjali Biswas Dietrich Klakow KELM 14 4 0 24 Jan 2022
LaMDA: Language Models for Dialog Applications R. Thoppilan Daniel De Freitas Jamie Hall Noam M. Shazeer Apoorv Kulshreshtha ... Blaise Aguera-Arcas Claire Cui M. Croak Ed H. Chi Quoc Le ALM 13 1,557 0 20 Jan 2022
Memory-assisted prompt editing to improve GPT-3 after deployment Aman Madaan Niket Tandon Peter Clark Yiming Yang KELM 19 0 0 16 Jan 2022
Reasoning Through Memorization: Nearest Neighbor Knowledge Graph Embeddings Peng Wang Xin Xie Xiaohan Wang Ningyu Zhang RALM 26 16 0 14 Jan 2022
DREAM: Improving Situational QA by First Elaborating the Situation Yuling Gu Bhavana Dalvi Peter Clark 19 18 0 16 Dec 2021
Learning To Retrieve Prompts for In-Context Learning Ohad Rubin Jonathan Herzig Jonathan Berant VPVLM RALM 14 664 0 16 Dec 2021
Rethinking Nearest Neighbors for Visual Classification Menglin Jia Bor-Chun Chen Zuxuan Wu Claire Cardie Serge J. Belongie Ser-Nam Lim SSL 25 10 0 15 Dec 2021
Faster Nearest Neighbor Machine Translation Shuhe Wang Jiwei Li Yuxian Meng Rongbin Ouyang Guoyin Wang Xiaoya Li Tianwei Zhang Shi Zong 22 12 0 15 Dec 2021
Improving and Diagnosing Knowledge-Based Visual Question Answering via Entity Enhanced Knowledge Injection Diego Garcia-Olano Yasumasa Onoe Joydeep Ghosh 6 18 0 13 Dec 2021
Improving language models by retrieving from trillions of tokens Sebastian Borgeaud A. Mensch Jordan Hoffmann Trevor Cai Eliza Rutherford ... Simon Osindero Karen Simonyan Jack W. Rae Erich Elsen Laurent Sifre KELM RALM 31 1,013 0 08 Dec 2021
How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN R. Thomas McCoy P. Smolensky Tal Linzen Jianfeng Gao Asli Celikyilmaz SyDa 17 119 0 18 Nov 2021
Personalized Federated Learning through Local Memorization Othmane Marfoq Giovanni Neglia Laetitia Kameni Richard Vidal FedML 20 87 0 17 Nov 2021
NeurInt : Learning to Interpolate through Neural ODEs Avinandan Bose Aniket Das Yatin Dandi P. Rai DiffM DRL 15 0 0 07 Nov 2021
Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling Renrui Zhang Rongyao Fang Wei Zhang Peng Gao Kunchang Li Jifeng Dai Yu Qiao Hongsheng Li VLM 184 385 0 06 Nov 2021
GNN-LM: Language Modeling based on Global Contexts via GNN Yuxian Meng Shi Zong Xiaoya Li Xiaofei Sun Tianwei Zhang Fei Wu Jiwei Li LRM 16 37 0 17 Oct 2021
Controllable Semantic Parsing via Retrieval Augmentation Panupong Pasupat Yuan Zhang Kelvin Guu 127 47 0 16 Oct 2021
Hindsight: Posterior-guided training of retrievers for improved open-ended generation Ashwin Paranjape Omar Khattab Christopher Potts Matei A. Zaharia Christopher D. Manning RALM 72 43 0 14 Oct 2021
The Inductive Bias of In-Context Learning: Rethinking Pretraining Example Design Yoav Levine Noam Wies Daniel Jannai D. Navon Yedid Hoshen Amnon Shashua AI4CE 11 36 0 09 Oct 2021
Capturing Structural Locality in Non-parametric Language Models Frank F. Xu Junxian He Graham Neubig Vincent J. Hellendoorn 11 14 0 06 Oct 2021
How BPE Affects Memorization in Transformers Eugene Kharitonov Marco Baroni Dieuwke Hupkes 161 32 0 06 Oct 2021
KNN-BERT: Fine-Tuning Pre-Trained Models with KNN Classifier Linyang Li Demin Song Ruotian Ma Xipeng Qiu Xuanjing Huang 27 21 0 06 Oct 2021
Data Augmentation Approaches in Natural Language Processing: A Survey Bohan Li Yutai Hou Wanxiang Che 119 270 0 05 Oct 2021
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding Hu Xu Gargi Ghosh Po-Yao (Bernie) Huang Dmytro Okhonko Armen Aghajanyan Florian Metze Luke Zettlemoyer Florian Metze Luke Zettlemoyer Christoph Feichtenhofer CLIP VLM 245 558 0 28 Sep 2021
Instance-Based Neural Dependency Parsing Hiroki Ouchi Jun Suzuki Sosuke Kobayashi Sho Yokoi Tatsuki Kuribayashi Masashi Yoshikawa Kentaro Inui 27 3 0 28 Sep 2021
Non-Parametric Online Learning from Human Feedback for Neural Machine Translation Dongqi Wang Hao-Ran Wei Zhirui Zhang Shujian Huang Jun Xie Jiajun Chen OffRL 44 15 0 23 Sep 2021
RETRONLU: Retrieval Augmented Task-Oriented Semantic Parsing Vivek Gupta Akshat Shrivastava Adithya Sagar Armen Aghajanyan Denis Savenkov RALM 29 21 0 21 Sep 2021
ConvFiT: Conversational Fine-Tuning of Pretrained Language Models Ivan Vulić Pei-hao Su Sam Coope D. Gerz Paweł Budzianowski I. Casanueva Nikola Mrkvsić Tsung-Hsien Wen 11 36 0 21 Sep 2021
Regularized Training of Nearest Neighbor Language Models Jean-François Ton Walter A. Talbott Shuangfei Zhai J. Susskind RALM 9 3 0 16 Sep 2021
Remember the context! ASR slot error correction through memorization Dhanush Bekal Ashish Shenoy Monica Sunkara S. Bodapati Katrin Kirchhoff KELM 21 12 0 10 Sep 2021
Efficient Nearest Neighbor Language Models Junxian He Graham Neubig Taylor Berg-Kirkpatrick RALM 191 103 0 09 Sep 2021