Generalization through Memorization: Nearest Neighbor Language Models

1 November 2019

Urvashi Khandelwal

Omer Levy

Dan Jurafsky

Luke Zettlemoyer

M. Lewis

RALM

ArXiv PDF HTML

Papers citing "Generalization through Memorization: Nearest Neighbor Language Models"

50 / 576 papers shown

Title
Heterogenous Memory Augmented Neural Networks Zihan Qiu Zhen Liu Shuicheng Yan Shanghang Zhang Jie Fu 13 0 0 17 Oct 2023
RegaVAE: A Retrieval-Augmented Gaussian Mixture Variational Auto-Encoder for Language Modeling Jingcheng Deng Liang Pang Huawei Shen Xueqi Cheng RALM 19 10 0 16 Oct 2023
Chameleon: a Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Models Wenqi Jiang Marco Zeller R. Waleffe Torsten Hoefler Gustavo Alonso 47 16 0 15 Oct 2023
InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining Boxin Wang Wei Ping Lawrence C. McAfee Peng-Tao Xu Bo Li M. Shoeybi Bryan Catanzaro RALM 16 45 0 11 Oct 2023
Goodtriever: Adaptive Toxicity Mitigation with Retrieval-augmented Models Luiza Amador Pozzobon B. Ermiş Patrick Lewis Sara Hooker 28 20 0 11 Oct 2023
How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advances Zihan Zhang Meng Fang Lingxi Chen Mohammad-Reza Namazi-Rad Jun Wang KELM 19 21 0 11 Oct 2023
A Meta-Learning Perspective on Transformers for Causal Language Modeling Xinbo Wu L. Varshney 23 6 0 09 Oct 2023
What do larger image classifiers memorise? Michal Lukasik Vaishnavh Nagarajan A. S. Rawat A. Menon Sanjiv Kumar 30 5 0 09 Oct 2023
Toolink: Linking Toolkit Creation and Using through Chain-of-Solving on Open-Source Model Cheng Qian Chenyan Xiong Zhenghao Liu Zhiyuan Liu LRM 24 12 0 08 Oct 2023
Self-Knowledge Guided Retrieval Augmentation for Large Language Models Yile Wang Peng Li Maosong Sun Yang Liu RALM KELM 29 42 0 08 Oct 2023
RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation Fangyuan Xu Weijia Shi Eunsol Choi RALM 27 143 0 06 Oct 2023
Retrieval meets Long Context Large Language Models Peng-Tao Xu Wei Ping Xianchao Wu Lawrence C. McAfee Chen Zhu Zihan Liu Sandeep Subramanian Evelina Bakhturina M. Shoeybi Bryan Catanzaro RALM LRM 14 79 0 04 Oct 2023
Dodo: Dynamic Contextual Compression for Decoder-only LMs Guanghui Qin Corby Rosset Ethan C. Chau Nikhil Rao Benjamin Van Durme 19 7 0 03 Oct 2023
OceanGPT: A Large Language Model for Ocean Science Tasks Zhen Bi Ningyu Zhang Yida Xue Yixin Ou Daxiong Ji Guozhou Zheng Huajun Chen ALM LLMAG 26 28 0 03 Oct 2023
RA-DIT: Retrieval-Augmented Dual Instruction Tuning Xi Victoria Lin Xilun Chen Mingda Chen Weijia Shi Maria Lomeli ... Jacob Kahn Gergely Szilvasy Mike Lewis Luke Zettlemoyer Scott Yih RALM 39 129 0 02 Oct 2023
Noise-Tolerant Unsupervised Adapter for Vision-Language Models Eman Ali Dayan Guan Muhammad Haris Khan Abdulmotaleb Elsaddik VLM 22 0 0 26 Sep 2023
Ragas: Automated Evaluation of Retrieval Augmented Generation ES Shahul Jithin James Luis Espinosa-Anke Steven Schockaert 80 175 0 26 Sep 2023
Can Whisper perform speech-based in-context learning? Siyin Wang Chao-Han Huck Yang Ji Wu Chao Zhang 19 24 0 13 Sep 2023
Towards Reliable and Fluent Large Language Models: Incorporating Feedback Learning Loops in QA Systems Dongyub Lee Taesun Whang Chanhee Lee Heuiseok Lim KELM 9 9 0 08 Sep 2023
ImageBind-LLM: Multi-modality Instruction Tuning Jiaming Han Renrui Zhang Wenqi Shao Peng Gao Peng-Tao Xu ... Yafei Wen Xiaoxin Chen Xiangyu Yue Hongsheng Li Yu Qiao MLLM 35 116 0 07 Sep 2023
RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model Fengxiang Bie Yibo Yang Zhongzhu Zhou Adam Ghanem Minjia Zhang ... Pareesa Ameneh Golnari David A. Clifton Yuxiong He Dacheng Tao S. Song EGVM 25 18 0 02 Sep 2023
RAMP: Retrieval-Augmented MOS Prediction via Confidence-based Dynamic Weighting Haibo Wang Shiwan Zhao Xiguang Zheng Yong Qin 21 11 0 31 Aug 2023
Supervised Contrastive Learning with Nearest Neighbor Search for Speech Emotion Recognition Xuechen Wang Shiwan Zhao Yong Qin 22 6 0 31 Aug 2023
LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Models Chi Han Qifan Wang Hao Peng Wenhan Xiong Yu Chen Heng Ji Sinong Wang 42 47 0 30 Aug 2023
Cross-Modal Retrieval Meets Inference:Improving Zero-Shot Classification with Cross-Modal Retrieval Seong-Hoon Eom Namgyu Ho Jaehoon Oh Se-Young Yun CLIP VLM 26 0 0 29 Aug 2023
CAGRA: Highly Parallel Graph Construction and Approximate Nearest Neighbor Search for GPUs Hiroyuki Ootomo Akira Naruse Corey J. Nolet Ray Wang Tamas B. Fehér Y. Wang GNN 20 20 0 29 Aug 2023
MEMORY-VQ: Compression for Tractable Internet-Scale Memory Yury Zemlyanskiy Michiel de Jong Luke Vilnis Santiago Ontañón William W. Cohen Sumit Sanghai Joshua Ainslie RALM MQ 33 0 0 28 Aug 2023
LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding Yushi Bai Xin Lv Jiajie Zhang Hong Lyu Jiankai Tang ... Aohan Zeng Lei Hou Yuxiao Dong Jie Tang Juanzi Li LLMAG RALM 26 492 0 28 Aug 2023
With a Little Help from your own Past: Prototypical Memory Networks for Image Captioning Manuele Barraco Sara Sarto Marcella Cornia Lorenzo Baraldi Rita Cucchiara VLM 51 19 0 23 Aug 2023
RSpell: Retrieval-augmented Framework for Domain Adaptive Chinese Spelling Check Siqi Song Qi Lv Lei Geng Ziqiang Cao G. Fu 17 5 0 16 Aug 2023
RAVEN: In-Context Learning with Retrieval-Augmented Encoder-Decoder Language Models Jie Huang Wei Ping Peng-Tao Xu M. Shoeybi Kevin Chen-Chuan Chang Bryan Catanzaro RALM 27 33 0 15 Aug 2023
WeaverBird: Empowering Financial Decision-Making with Large Language Model, Knowledge Base, and Search Engine Siqiao Xue Fan Zhou Y. Xu Ming Jin Qingsong Wen ... Jun Zhou Shuo Xie D. Xiu James Y. Zhang Hongyuan Mei RALM AIFin 31 15 0 10 Aug 2023
SILO Language Models: Isolating Legal Risk In a Nonparametric Datastore Sewon Min Suchin Gururangan Eric Wallace Hannaneh Hajishirzi Noah A. Smith Luke Zettlemoyer AILaw 22 63 0 08 Aug 2023
TabR: Tabular Deep Learning Meets Nearest Neighbors in 2023 Yu. V. Gorishniy Ivan Rubachev Nikolay Kartashev Daniil Shlenskii Akim Kotelnikov Artem Babenko OOD LMTD 11 13 0 26 Jul 2023
Benchmarking and Analyzing Generative Data for Visual Recognition Bo-wen Li Haotian Liu Liangyu Chen Yong Jae Lee C. Li Ziwei Liu EGVM VLM 16 4 0 25 Jul 2023
Learning to Retrieve In-Context Examples for Large Language Models Liang Wang Nan Yang Furu Wei RALM 30 38 0 14 Jul 2023
Generating Benchmarks for Factuality Evaluation of Language Models Dor Muhlgay Ori Ram Inbal Magar Yoav Levine Nir Ratner Yonatan Belinkov Omri Abend Kevin Leyton-Brown Amnon Shashua Y. Shoham HILM 25 91 0 13 Jul 2023
Copy Is All You Need Tian Lan Deng Cai Yan Wang Heyan Huang Xian-Ling Mao 27 26 0 13 Jul 2023
Pluggable Neural Machine Translation Models via Memory-augmented Adapters Yuzhuang Xu Shuo Wang Peng Li Xuebo Liu Xiaolong Wang Weidong Liu Yang Liu 27 1 0 12 Jul 2023
ReLoRA: High-Rank Training Through Low-Rank Updates Vladislav Lialin Namrata Shivagunde Sherin Muckatira Anna Rumshisky BDL 29 93 0 11 Jul 2023
Linear Alignment of Vision-language Models for Image Captioning Fabian Paischer M. Hofmarcher Sepp Hochreiter Thomas Adler CLIP VLM 42 0 0 10 Jul 2023
Focused Transformer: Contrastive Training for Context Scaling Szymon Tworkowski Konrad Staniszewski Mikolaj Pacek Yuhuai Wu Henryk Michalewski Piotr Milo's 21 135 0 06 Jul 2023
Multimodal Prompt Retrieval for Generative Visual Question Answering Timothy Ossowski Junjie Hu 13 1 0 30 Jun 2023
LeanDojo: Theorem Proving with Retrieval-Augmented Language Models Kaiyu Yang Aidan M. Swope Alex Gu Rahul Chalamala Peiyang Song Shixing Yu Saad Godil R. Prenger Anima Anandkumar RALM 19 207 0 27 Jun 2023
Long-range Language Modeling with Self-retrieval Ohad Rubin Jonathan Berant RALM KELM 19 18 0 23 Jun 2023
RepoFusion: Training Code Models to Understand Your Repository Disha Shrivastava Denis Kocetkov H. D. Vries Dzmitry Bahdanau Torsten Scholak 81 27 0 19 Jun 2023
Guiding Language Models of Code with Global Context using Monitors Lakshya A Agrawal Aditya Kanade Navin Goyal Shuvendu K. Lahiri S. Rajamani 38 23 0 19 Jun 2023
GLIMMER: generalized late-interaction memory reranker Michiel de Jong Yury Zemlyanskiy Nicholas FitzGerald Sumit Sanghai William W. Cohen Joshua Ainslie RALM 41 4 0 17 Jun 2023
Neural Priming for Sample-Efficient Adaptation Matthew Wallingford Vivek Ramanujan Alex Fang Aditya Kusupati Roozbeh Mottaghi Aniruddha Kembhavi Ludwig Schmidt Ali Farhadi VLM 108 13 0 16 Jun 2023
Encyclopedic VQA: Visual questions about detailed properties of fine-grained categories Thomas Mensink J. Uijlings Lluis Castrejon A. Goel Felipe Cadar Howard Zhou Fei Sha A. Araújo V. Ferrari 34 37 0 15 Jun 2023