Generalization through Memorization: Nearest Neighbor Language Models

1 November 2019

Urvashi Khandelwal

Omer Levy

Dan Jurafsky

Luke Zettlemoyer

M. Lewis

RALM

ArXiv PDF HTML

Papers citing "Generalization through Memorization: Nearest Neighbor Language Models"

50 / 576 papers shown

Title
Retrieval-Enhanced Contrastive Vision-Text Models Ahmet Iscen Mathilde Caron Alireza Fathi Cordelia Schmid CLIP VLM 25 26 0 12 Jun 2023
Retrosynthesis Prediction with Local Template Retrieval Shufang Xie Rui Yan Junliang Guo Yingce Xia Lijun Wu Tao Qin 24 12 0 07 Jun 2023
Information Flow Control in Machine Learning through Modular Model Architecture Trishita Tiwari Suchin Gururangan Chuan Guo Weizhe Hua Sanjay Kariyappa Udit Gupta Wenjie Xiong Kiwan Maeng Hsien-Hsin S. Lee G. E. Suh 19 6 0 05 Jun 2023
Retrieval-Enhanced Visual Prompt Learning for Few-shot Classification Jintao Rong Hao Chen Tianrun Chen Linlin Ou Xinyi Yu Yifan Liu VLM VPVLM 13 6 0 04 Jun 2023
The Information Pathways Hypothesis: Transformers are Dynamic Self-Ensembles Md Shamim Hussain Mohammed J. Zaki D. Subramanian 37 3 0 02 Jun 2023
Exposing Attention Glitches with Flip-Flop Language Modeling Bingbin Liu Jordan T. Ash Surbhi Goel A. Krishnamurthy Cyril Zhang LRM 27 46 0 01 Jun 2023
LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting R. Ramos Bruno Martins Desmond Elliott VLM 13 16 0 31 May 2023
Information Association for Language Model Updating by Mitigating LM-Logical Discrepancy Pengfei Yu Heng Ji KELM 31 9 0 29 May 2023
Test-Time Training on Nearest Neighbors for Large Language Models Moritz Hardt Yu Sun VLM RALM 33 19 0 29 May 2023
Prompt-Guided Retrieval Augmentation for Non-Knowledge-Intensive Tasks Zhicheng Guo Sijie Cheng Yile Wang Peng Li Yang Liu RALM 22 18 0 28 May 2023
Landmark Attention: Random-Access Infinite Context Length for Transformers Amirkeivan Mohtashami Martin Jaggi LLMAG 19 149 0 25 May 2023
Surface-Based Retrieval Reduces Perplexity of Retrieval-Augmented Language Models Ehsan Doostmohammadi Tobias Norlund Marco Kuhlmann Richard Johansson RALM 25 7 0 25 May 2023
Enhancing Grammatical Error Correction Systems with Explanations Yuejiao Fei Leyang Cui Sen Yang Wai Lam Zhenzhong Lan Shuming Shi 26 13 0 25 May 2023
Aligning Language Models to User Opinions EunJeong Hwang Bodhisattwa Prasad Majumder Niket Tandon 27 59 0 24 May 2023
Privacy Implications of Retrieval-Based Language Models Yangsibo Huang Samyak Gupta Zexuan Zhong K. Li Danqi Chen RALM 25 29 0 24 May 2023
Adapting Language Models to Compress Contexts Alexis Chevalier Alexander Wettig Anirudh Ajith Danqi Chen LLMAG 11 173 0 24 May 2023
Trusting Your Evidence: Hallucinate Less with Context-aware Decoding Weijia Shi Xiaochuang Han M. Lewis Yulia Tsvetkov Luke Zettlemoyer Scott Yih HILM 19 187 0 24 May 2023
Enabling Large Language Models to Generate Text with Citations Tianyu Gao Howard Yen Jiatong Yu Danqi Chen LM&MA HILM 29 311 0 24 May 2023
KNN-LM Does Not Improve Open-ended Text Generation Shufan Wang Yixiao Song Andrew Drozdov Aparna Garimella Varun Manjunatha Mohit Iyyer RALM 17 8 0 24 May 2023
Accessing Higher Dimensions for Unsupervised Word Translation Sida I. Wang 28 0 0 23 May 2023
To Copy Rather Than Memorize: A Vertical Learning Paradigm for Knowledge Graph Completion Rui Li Xu Chen Chaozhuo Li Yanming Shen Jianan Zhao ... Weihao Han Hao-Lun Sun Weiwei Deng Qi Zhang Xing Xie 21 7 0 23 May 2023
Non-parametric, Nearest-neighbor-assisted Fine-tuning for Neural Machine Translation Jiayi Wang Ke Min Wang Yuqi Zhang Yu Zhao Pontus Stenetorp 11 0 0 23 May 2023
Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts Jian Xie Kai Zhang Jiangjie Chen Renze Lou Yu-Chuan Su RALM 198 153 0 22 May 2023
LM vs LM: Detecting Factual Errors via Cross Examination Roi Cohen May Hamri Mor Geva Amir Globerson HILM 29 117 0 22 May 2023
Nearest Neighbor Machine Translation is Meta-Optimizer on Output Projection Layer R. Gao Zhirui Zhang Yichao Du Lemao Liu Rui Wang 13 2 0 22 May 2023
Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Models Oana Ignat Zhijing Jin Artem Abzaliev Laura Biester Santiago Castro ... Verónica Pérez-Rosas Siqi Shen Zekun Wang Winston Wu Rada Mihalcea LRM 29 6 0 21 May 2023
Description-Based Text Similarity Shauli Ravfogel Valentina Pyatkin Amir D. N. Cohen Avshalom Manevich Yoav Goldberg 20 5 0 21 May 2023
Complex Claim Verification with Evidence Retrieved in the Wild Jifan Chen Grace Kim Aniruddh Sriram Greg Durrett Eunsol Choi HILM 22 68 0 19 May 2023
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing Zhibin Gou Zhihong Shao Yeyun Gong Yelong Shen Yujiu Yang Nan Duan Weizhu Chen KELM LRM 36 357 0 19 May 2023
Decouple knowledge from parameters for plug-and-play language modeling Xin Cheng Yankai Lin Xiuying Chen Dongyan Zhao Rui Yan KELM 27 2 0 19 May 2023
ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval Yue Yu Yuchen Zhuang Rongzhi Zhang Yu Meng Jiaming Shen Chao Zhang VLM 32 33 0 18 May 2023
Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language Models Shangbin Feng Weijia Shi Yuyang Bai Vidhisha Balachandran Tianxing He Yulia Tsvetkov KELM 45 28 0 17 May 2023
Memorization for Good: Encryption with Autoregressive Language Models Samuel Stevens Yung-Chun Su 15 0 0 15 May 2023
Synergistic Interplay between Search and Large Language Models for Information Retrieval Jiazhan Feng Chongyang Tao Xiubo Geng Tao Shen Can Xu Guodong Long Dongyan Zhao Daxin Jiang KELM 55 5 0 12 May 2023
Active Retrieval Augmented Generation Zhengbao Jiang Frank F. Xu Luyu Gao Zhiqing Sun Qian Liu Jane Dwivedi-Yu Yiming Yang Jamie Callan Graham Neubig RALM 9 252 0 11 May 2023
HistAlign: Improving Context Dependency in Language Generation by Aligning with History David Wan Shiyue Zhang Mohit Bansal AI4TS 32 5 0 08 May 2023
DAMO-NLP at SemEval-2023 Task 2: A Unified Retrieval-augmented System for Multilingual Named Entity Recognition Zeqi Tan Shen Huang Zixia Jia Jiong Cai Yinghui Li ... Y. Zhuang Kewei Tu Pengjun Xie Fei Huang Yong-jia Jiang 33 8 0 05 May 2023
Now It Sounds Like You: Learning Personalized Vocabulary On Device Sida Wang Ashish Shenoy P. Chuang John Nguyen VLM 30 2 0 05 May 2023
Re $^3$ Dial: Retrieve, Reorganize and Rescale Dialogue Corpus for Long-Turn Open-Domain Dialogue Pre-training Jiaxin Wen Hao Zhou Jian-Yu Guan Minlie Huang 28 0 0 04 May 2023
Lift Yourself Up: Retrieval-augmented Text Generation with Self Memory Xin Cheng Di Luo Xiuying Chen Lemao Liu Dongyan Zhao Rui Yan RALM 145 91 0 03 May 2023
Unlimiformer: Long-Range Transformers with Unlimited Length Input Amanda Bertsch Uri Alon Graham Neubig Matthew R. Gormley RALM 94 122 0 02 May 2023
Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4 Kent K. Chang Mackenzie Cramer Sandeep Soni David Bamman RALM 143 110 0 28 Apr 2023
Analogy-Forming Transformers for Few-Shot 3D Parsing N. Gkanatsios M. Singh Zhaoyuan Fang Shubham Tulsiani Katerina Fragkiadaki 3DPC 3DV 19 2 0 27 Apr 2023
Revisiting k-NN for Fine-tuning Pre-trained Language Models Lei Li Jing Chen Bo Tian Ning Zhang 19 1 0 18 Apr 2023
Tool Learning with Foundation Models Yujia Qin Shengding Hu Yankai Lin Weize Chen Ning Ding ... Cheng Yang Tongshuang Wu Heng Ji Zhiyuan Liu Maosong Sun 42 199 0 17 Apr 2023
Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study Boxin Wang Wei Ping P. Xu Lawrence C. McAfee Zihan Liu ... Oleksii Kuchaiev Bo-wen Li Chaowei Xiao Anima Anandkumar Bryan Catanzaro RALM 38 55 0 13 Apr 2023
Improving Image Recognition by Retrieving from Web-Scale Image-Text Data Ahmet Iscen Alireza Fathi Cordelia Schmid VLM 3DV 33 25 0 11 Apr 2023
Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis Wenhao Zhu Hongyi Liu Qingxiu Dong Jingjing Xu Shujian Huang Lingpeng Kong Jiajun Chen Lei Li LRM 29 139 0 10 Apr 2023
LADER: Log-Augmented DEnse Retrieval for Biomedical Literature Search Qiao Jin Andrew Shin Zhiyong Lu RALM 22 7 0 10 Apr 2023
ConceptEVA: Concept-Based Interactive Exploration and Customization of Document Summaries Xiaoyu Zhang J. Li Po-Wei Chi Senthil K. Chandrasegaran Kwan-Liu Ma 4 22 0 31 Mar 2023