Generalization through Memorization: Nearest Neighbor Language Models

1 November 2019

Urvashi Khandelwal

Omer Levy

Dan Jurafsky

Luke Zettlemoyer

M. Lewis

RALM

ArXiv PDF HTML

Papers citing "Generalization through Memorization: Nearest Neighbor Language Models"

50 / 576 papers shown

Title
A Survey of Knowledge Enhanced Pre-trained Language Models Linmei Hu Zeyi Liu Ziwang Zhao Lei Hou Liqiang Nie Juanzi Li KELM VLM 17 121 0 11 Nov 2022
Suffix Retrieval-Augmented Language Modeling Zecheng Wang Yik-Cheung Tam RALM 13 1 0 06 Nov 2022
Exemplar Guided Deep Neural Network for Spatial Transcriptomics Analysis of Gene Expression Prediction Yan Yang Md. Zakir Hossain Eric A. Stone Shafin Rahman AI4TS 21 14 0 30 Oct 2022
Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models Xiaoman Pan Wenlin Yao Hongming Zhang Dian Yu Dong Yu Jianshu Chen KELM 189 22 0 28 Oct 2022
You can't pick your neighbors, or can you? When and how to rely on retrieval in the $k$ NN-LM Andrew Drozdov Shufan Wang Razieh Rahimi Andrew McCallum Hamed Zamani Mohit Iyyer RALM 105 17 0 28 Oct 2022
Nearest Neighbor Language Models for Stylistic Controllable Generation Severino Trotta Lucie Flek Charles F Welch 10 4 0 27 Oct 2022
EUREKA: EUphemism Recognition Enhanced through Knn-based methods and Augmentation Sedrick Scott Keh Rohit K Bharadwaj Emmy Liu Simone Tedeschi Varun Gangal Roberto Navigli 11 7 0 23 Oct 2022
Generative Knowledge Graph Construction: A Review Hongbin Ye Ningyu Zhang Hui Chen Huajun Chen 43 70 0 23 Oct 2022
Cross-domain Generalization for AMR Parsing Xuefeng Bai Sen Yang Leyang Cui Linfeng Song Yue Zhang 38 2 0 22 Oct 2022
Enhancing Tabular Reasoning with Pattern Exploiting Training Abhilash Shankarampeta Vivek Gupta Shuo Zhang LMTD RALM ReLM 58 6 0 21 Oct 2022
Rescue Implicit and Long-tail Cases: Nearest Neighbor Relation Extraction Zhen Wan Qianying Liu Zhuoyuan Mao Fei Cheng Sadao Kurohashi Jiwei Li 16 8 0 21 Oct 2022
Schema-aware Reference as Prompt Improves Data-Efficient Knowledge Graph Construction Yunzhi Yao Shengyu Mao Ningyu Zhang Xiangnan Chen Shumin Deng Xi Chen Huajun Chen 26 9 0 19 Oct 2022
Transferring Knowledge via Neighborhood-Aware Optimal Transport for Low-Resource Hate Speech Detection Tulika Bose Irina Illina Dominique Fohr 13 0 0 17 Oct 2022
RARR: Researching and Revising What Language Models Say, Using Language Models Luyu Gao Zhuyun Dai Panupong Pasupat Anthony Chen Arun Tejasvi Chaganty ... Vincent Zhao Ni Lao Hongrae Lee Da-Cheng Juan Kelvin Guu HILM KELM 33 256 0 17 Oct 2022
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey Sachin Kumar Vidhisha Balachandran Lucille Njoo Antonios Anastasopoulos Yulia Tsvetkov ELM 66 85 0 14 Oct 2022
Decoupled Context Processing for Context Augmented Language Modeling Zonglin Li Ruiqi Guo Surinder Kumar RALM KELM 14 22 0 11 Oct 2022
Measuring and Narrowing the Compositionality Gap in Language Models Ofir Press Muru Zhang Sewon Min Ludwig Schmidt Noah A. Smith M. Lewis ReLM KELM LRM 46 550 0 07 Oct 2022
MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text Wenhu Chen Hexiang Hu Xi Chen Pat Verga William W. Cohen RALM 8 141 0 06 Oct 2022
Nonparametric Decoding for Generative Retrieval Hyunji Lee Jaeyoung Kim Hoyeon Chang Hanseok Oh Sohee Yang Vladimir Karpukhin Yi Lu Minjoon Seo RALM 19 5 0 05 Oct 2022
Memory in humans and deep language models: Linking hypotheses for model augmentation Omri Raccah Pheobe Chen Ted Willke David Poeppel Vy A. Vo RALM 13 1 0 04 Oct 2022
Zemi: Learning Zero-Shot Semi-Parametric Language Models from Multiple Tasks Zhenhailong Wang Xiaoman Pan Dian Yu Dong Yu Jianshu Chen Heng Ji VLM 38 9 0 01 Oct 2022
Re-Imagen: Retrieval-Augmented Text-to-Image Generator Wenhu Chen Hexiang Hu Chitwan Saharia William W. Cohen VLM 117 161 0 29 Sep 2022
Non-Parametric Temporal Adaptation for Social Media Topic Classification Fatemehsadat Mireshghallah Nikolai Vogler Junxian He Omar U. Florez Ahmed El-Kishky Taylor Berg-Kirkpatrick TTA 9 0 0 13 Sep 2022
A Review of Sparse Expert Models in Deep Learning W. Fedus J. Dean Barret Zoph MoE 13 144 0 04 Sep 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 28 109 0 31 Aug 2022
Fusing Sentence Embeddings Into LSTM-based Autoregressive Language Models Vilém Zouhar Marius Mosbach Dietrich Klakow 24 1 0 04 Aug 2022
Retrieval-Augmented Transformer for Image Captioning Sara Sarto Marcella Cornia Lorenzo Baraldi Rita Cucchiara 19 57 0 26 Jul 2022
Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Models Robin Rombach A. Blattmann Bjorn Ommer DiffM 16 70 0 26 Jul 2022
MQRetNN: Multi-Horizon Time Series Forecasting with Retrieval Augmentation Sitan Yang Carson Eisenach Dhruv Madeka AI4TS 22 7 0 21 Jul 2022
Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification Renrui Zhang Zhang Wei Rongyao Fang Peng Gao Kunchang Li Jifeng Dai Yu Qiao Hongsheng Li VLM 19 292 0 19 Jul 2022
N-Grammer: Augmenting Transformers with latent n-grams Aurko Roy Rohan Anil Guangda Lai Benjamin Lee Jeffrey Zhao ... Yu Phuong Dao Christopher Fifty Z. Chen Yonghui Wu 11 6 0 13 Jul 2022
Repository-Level Prompt Generation for Large Language Models of Code Disha Shrivastava Hugo Larochelle Daniel Tarlow 15 137 0 26 Jun 2022
Memory-Based Model Editing at Scale E. Mitchell Charles Lin Antoine Bosselut Christopher D. Manning Chelsea Finn KELM 22 318 0 13 Jun 2022
Annotation Error Detection: Analyzing the Past and Present for a More Coherent Future Jan-Christoph Klie Bonnie Webber Iryna Gurevych 40 43 0 05 Jun 2022
Dict-TTS: Learning to Pronounce with Prior Dictionary Knowledge for Text-to-Speech Ziyue Jiang Zhe Su Zhou Zhao Qian Yang Yi Ren Jinglin Liu Zhe Ye 24 4 0 05 Jun 2022
Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning Xiang Chen Lei Li Ningyu Zhang Xiaozhuan Liang Shumin Deng Chuanqi Tan Fei Huang Luo Si Huajun Chen VLM 23 52 0 29 May 2022
kNN-Prompt: Nearest Neighbor Zero-Shot Inference Weijia Shi Julian Michael Suchin Gururangan Luke Zettlemoyer RALM VLM 13 32 0 27 May 2022
Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval Pascal Notin M. Dias J. Frazer Javier Marchena-Hurtado Aidan N. Gomez D. Marks Y. Gal 53 176 0 27 May 2022
Training Language Models with Memory Augmentation Zexuan Zhong Tao Lei Danqi Chen RALM 232 127 0 25 May 2022
ORCA: Interpreting Prompted Language Models via Locating Supporting Data Evidence in the Ocean of Pretraining Data Xiaochuang Han Yulia Tsvetkov 11 27 0 25 May 2022
Chunk-based Nearest Neighbor Machine Translation Pedro Henrique Martins Zita Marinho André F.T. Martins RALM 78 28 0 24 May 2022
StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Models Adam Livska Tomávs Kovciský E. Gribovskaya Tayfun Terzi Eren Sezener ... Susannah Young Ellen Gilsenan-McMahon Sophia Austin Phil Blunsom Angeliki Lazaridou KELM 232 90 0 23 May 2022
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models Kushal Tirumala Aram H. Markosyan Luke Zettlemoyer Armen Aghajanyan TDI 16 185 0 22 May 2022
RankGen: Improving Text Generation with Large Ranking Models Kalpesh Krishna Yapei Chang John Wieting Mohit Iyyer AIMat 16 68 0 19 May 2022
Long-term Control for Dialogue Generation: Methods and Evaluation Ramya Ramakrishnan H. Narangodage M. Schilman Kilian Q. Weinberger Ryan T. McDonald 11 8 0 15 May 2022
Near-Negative Distinction: Giving a Second Life to Human Evaluation Datasets Philippe Laban Chien-Sheng Wu Wenhao Liu Caiming Xiong 33 5 0 13 May 2022
kNN-Embed: Locally Smoothed Embedding Mixtures For Multi-interest Candidate Retrieval Ahmed El-Kishky Thomas Markovich Kenny Leung Frank Portman A. Haghighi Ying Xiao 11 12 0 12 May 2022
Few-shot Mining of Naturally Occurring Inputs and Outputs Mandar Joshi Terra Blevins M. Lewis Daniel S. Weld Luke Zettlemoyer 25 1 0 09 May 2022
Relation Extraction as Open-book Examination: Retrieval-enhanced Prompt Tuning Xiang Chen Lei Li Ningyu Zhang Chuanqi Tan Fei Huang Luo Si Huajun Chen RALM VLM 24 36 0 04 May 2022
Retrieval-Enhanced Machine Learning Hamed Zamani Fernando Diaz Mostafa Dehghani Donald Metzler Michael Bendersky 11 49 0 02 May 2022