Generalization through Memorization: Nearest Neighbor Language Models

1 November 2019

Urvashi Khandelwal

Omer Levy

Dan Jurafsky

Luke Zettlemoyer

M. Lewis

RALM

ArXiv PDF HTML

Papers citing "Generalization through Memorization: Nearest Neighbor Language Models"

50 / 576 papers shown

Title
MoViT: Memorizing Vision Transformers for Medical Image Analysis Yiqing Shen Pengfei Guo Jinpu Wu Qi Huang Nhat Le Jinyuan Zhou Shanshan Jiang Mathias Unberath ViT MedIm 18 10 0 27 Mar 2023
Scaling Expert Language Models with Unsupervised Domain Discovery Suchin Gururangan Margaret Li M. Lewis Weijia Shi Tim Althoff Noah A. Smith Luke Zettlemoyer MoE 17 46 0 24 Mar 2023
$k$ NN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inference Benfeng Xu Quan Wang Zhendong Mao Yajuan Lyu Qiaoqiao She Yongdong Zhang 93 52 0 24 Mar 2023
Retrieval-Augmented Classification with Decoupled Representation Xinnian Liang Shuangzhi Wu Hui Huang Jiaqi Bai Chao Bian Zhoujun Li 13 0 0 23 Mar 2023
Language Model Behavior: A Comprehensive Survey Tyler A. Chang Benjamin Bergen VLM LRM LM&MA 27 103 0 20 Mar 2023
On-the-fly Text Retrieval for End-to-End ASR Adaptation Bolaji Yusuf Aditya Gourav Ankur Gandhe I. Bulyko KELM RALM 27 4 0 20 Mar 2023
Text-to-image Diffusion Models in Generative AI: A Survey Chenshuang Zhang Chaoning Zhang Mengchun Zhang In So Kweon VLM 47 264 0 14 Mar 2023
MetaTroll: Few-shot Detection of State-Sponsored Trolls with Transformer Adapters Lin Tian Xiuzhen Zhang Jey Han Lau 22 10 0 13 Mar 2023
A Theoretical Analysis Of Nearest Neighbor Search On Approximate Near Neighbor Graph Anshumali Shrivastava Zhao-quan Song Zhaozhuo Xu GNN 9 8 0 10 Mar 2023
Semiparametric Language Models Are Scalable Continual Learners Guangyue Peng Tao Ge Si-Qing Chen Furu Wei Houfeng Wang KELM 39 10 0 02 Mar 2023
Nearest Neighbors Meet Deep Neural Networks for Point Cloud Analysis Renrui Zhang Liuhui Wang Ziyu Guo Jianbo Shi 3DPC 32 10 0 01 Mar 2023
Retrieved Sequence Augmentation for Protein Representation Learning Chang Ma Haiteng Zhao Lin Zheng Jiayi Xin Qintong Li Lijun Wu Zhihong Deng Yang Lu Qi Liu Lingpeng Kong AI4TS 20 9 0 24 Feb 2023
Federated Nearest Neighbor Machine Translation Yichao Du Zhirui Zhang Bingzhe Wu Lemao Liu Tong Bill Xu Enhong Chen FedML 21 6 0 23 Feb 2023
Simple and Scalable Nearest Neighbor Machine Translation Yu-Hsiu Dai Zhirui Zhang Qiuzhi Liu Qu Cui Wei-Hong Li Yichao Du Tong Bill Xu 24 16 0 23 Feb 2023
On the Generalization Ability of Retrieval-Enhanced Transformers Tobias Norlund Ehsan Doostmohammadi Richard Johansson Marco Kuhlmann RALM 24 6 0 23 Feb 2023
$k$ NN-Adapter: Efficient Domain Adaptation for Black-Box Language Models Yangsibo Huang Daogao Liu Zexuan Zhong Weijia Shi Y. Lee RALM ALM 12 14 0 21 Feb 2023
Retrieval-augmented Image Captioning R. Ramos Desmond Elliott Bruno Martins VLM 22 29 0 16 Feb 2023
Augmented Language Models: a Survey Grégoire Mialon Roberto Dessì Maria Lomeli Christoforos Nalmpantis Ramakanth Pasunuru ... Jane Dwivedi-Yu Asli Celikyilmaz Edouard Grave Yann LeCun Thomas Scialom LRM KELM 31 366 0 15 Feb 2023
Augmenting Zero-Shot Dense Retrievers with Plug-in Mixture-of-Memories Suyu Ge Chenyan Xiong Corby Rosset Arnold Overwijk Jiawei Han Paul N. Bennett VLM 33 6 0 07 Feb 2023
ResMem: Learn what you can and memorize the rest Zitong Yang Michal Lukasik Vaishnavh Nagarajan Zong-xiao Li A. S. Rawat Manzil Zaheer A. Menon Surinder Kumar VLM 27 8 0 03 Feb 2023
In-Context Retrieval-Augmented Language Models Ori Ram Yoav Levine Itay Dalmedigos Dor Muhlgay Amnon Shashua Kevin Leyton-Brown Y. Shoham KELM RALM LRM 15 536 0 31 Jan 2023
N-Gram Nearest Neighbor Machine Translation Rui Lv Junliang Guo Rui Wang Xu Tan Qi Liu Tao Qin 23 2 0 30 Jan 2023
REPLUG: Retrieval-Augmented Black-Box Language Models Weijia Shi Sewon Min Michihiro Yasunaga Minjoon Seo Rich James M. Lewis Luke Zettlemoyer Wen-tau Yih RALM VLM KELM 51 575 0 30 Jan 2023
ProtST: Multi-Modality Learning of Protein Sequences and Biomedical Texts Minghao Xu Xinyu Yuan Santiago Miret Jian Tang AI4TS 25 94 0 28 Jan 2023
Semi-Parametric Video-Grounded Text Generation Sungdong Kim Jin-Hwa Kim Jiyoung Lee Minjoon Seo VGen 22 14 0 27 Jan 2023
Pre-computed memory or on-the-fly encoding? A hybrid approach to retrieval augmentation makes the most of your compute Michiel de Jong Yury Zemlyanskiy Nicholas FitzGerald Joshua Ainslie Sumit Sanghai Fei Sha William W. Cohen RALM 27 16 0 25 Jan 2023
Learning Customized Visual Models with Retrieval-Augmented Knowledge Haotian Liu Kilho Son Jianwei Yang Ce Liu Jianfeng Gao Yong Jae Lee Chunyuan Li VLM 38 53 0 17 Jan 2023
Structured Case-based Reasoning for Inference-time Adaptation of Text-to-SQL parsers Abhijeet Awasthi Soumen Chakrabarti Sunita Sarawagi 14 5 0 10 Jan 2023
Why do Nearest Neighbor Language Models Work? Frank F. Xu Uri Alon Graham Neubig RALM 18 21 0 07 Jan 2023
Automating Nearest Neighbor Search Configuration with Constrained Optimization Philip Sun Ruiqi Guo Surinder Kumar 15 7 0 04 Jan 2023
Analogical Inference Enhanced Knowledge Graph Embedding Zhen Yao Wen Zhang Mingyang Chen Yufen Huang Yezhou Yang Hua-zeng Chen 36 12 0 03 Jan 2023
Rethinking with Retrieval: Faithful Large Language Model Inference Hangfeng He Hongming Zhang Dan Roth KELM LRM 141 156 0 31 Dec 2022
Continual Contrastive Finetuning Improves Low-Resource Relation Extraction Wenxuan Zhou Sheng Zhang Tristan Naumann Muhao Chen Hoifung Poon 43 6 0 21 Dec 2022
When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories Alex Troy Mallen Akari Asai Victor Zhong Rajarshi Das Daniel Khashabi Hannaneh Hajishirzi RALM HILM KELM 35 511 0 20 Dec 2022
Empowering Sentence Encoders with Prompting and Label Retrieval for Zero-shot Text Classification Jimin Hong Jungsoo Park Daeyoung Kim Seongjae Choi Bokyung Son Jaewoo Kang 19 3 0 20 Dec 2022
CoCoMIC: Code Completion By Jointly Modeling In-file and Cross-file Context Yangruibo Ding Zijian Wang Wasi Uddin Ahmad M. K. Ramanathan Ramesh Nallapati Parminder Bhatia Dan Roth Bing Xiang 16 68 0 20 Dec 2022
Training Trajectories of Language Models Across Scales Mengzhou Xia Mikel Artetxe Chunting Zhou Xi Victoria Lin Ramakanth Pasunuru Danqi Chen Luke Zettlemoyer Ves Stoyanov AIFin LRM 31 53 0 19 Dec 2022
Can Retriever-Augmented Language Models Reason? The Blame Game Between the Retriever and the Language Model Parishad BehnamGhader Santiago Miret Siva Reddy ReLM LRM 14 36 0 18 Dec 2022
Evaluating Step-by-Step Reasoning through Symbolic Verification Yi-Fan Zhang Hanlin Zhang Li Erran Li Eric P. Xing ReLM LRM 11 8 0 16 Dec 2022
Memories are One-to-Many Mapping Alleviators in Talking Face Generation Anni Tang Tianyu He Xuejiao Tan Jun Ling Liang Song CVBM 23 23 0 09 Dec 2022
Document-Level Abstractive Summarization Gonçalo Raposo Afonso Raposo Ana Sofia Carmo 19 1 0 06 Dec 2022
Meta-Learning Fast Weight Language Models Kevin Clark Kelvin Guu Ming-Wei Chang Panupong Pasupat Geoffrey E. Hinton Mohammad Norouzi KELM 27 13 0 05 Dec 2022
Retrieval as Attention: End-to-end Learning of Retrieval and Reading within a Single Transformer Zhengbao Jiang Luyu Gao Jun Araki Haibo Ding Zhiruo Wang Jamie Callan Graham Neubig RALM 25 40 0 05 Dec 2022
GNN-SL: Sequence Labeling Based on Nearest Examples via GNN Shuhe Wang Yuxian Meng Rongbin Ouyang Jiwei Li Tianwei Zhang Lingjuan Lyu Guoyin Wang 19 9 0 05 Dec 2022
Nonparametric Masked Language Modeling Sewon Min Weijia Shi M. Lewis Xilun Chen Wen-tau Yih Hannaneh Hajishirzi Luke Zettlemoyer RALM 40 48 0 02 Dec 2022
Data-Efficient Finetuning Using Cross-Task Nearest Neighbors Hamish Ivison Noah A. Smith Hannaneh Hajishirzi Pradeep Dasigi 31 19 0 01 Dec 2022
Task-Specific Embeddings for Ante-Hoc Explainable Text Classification Kishaloy Halder Josip Krapac A. Akbik Anthony Brew Matti Lyra 30 0 0 30 Nov 2022
Retrieval-Augmented Multimodal Language Modeling Michihiro Yasunaga Armen Aghajanyan Weijia Shi Rich James J. Leskovec Percy Liang M. Lewis Luke Zettlemoyer Wen-tau Yih RALM 11 95 0 22 Nov 2022
Token Turing Machines Michael S. Ryoo K. Gopalakrishnan Kumara Kahatapitiya Ted Xiao Kanishka Rao Austin Stone Yao Lu Julian Ibarz Anurag Arnab 27 21 0 16 Nov 2022
Error-Robust Retrieval for Chinese Spelling Check Xunjian Yin Xinyu Hu Jin Jiang Xiao-Yi Wan 20 3 0 15 Nov 2022