Generalization through Memorization: Nearest Neighbor Language Models

1 November 2019

Urvashi Khandelwal

Omer Levy

Dan Jurafsky

Luke Zettlemoyer

M. Lewis

RALM

ArXiv PDF HTML

Papers citing "Generalization through Memorization: Nearest Neighbor Language Models"

50 / 576 papers shown

Title
Operationalizing the Blueprint for an AI Bill of Rights: Recommendations for Practitioners, Researchers, and Policy Makers Alex Oesterling Usha Bhalla Suresh Venkatasubramanian Himabindu Lakkaraju 41 1 0 11 Jul 2024
Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting Zilong Wang Zifeng Wang Long Le Huaixiu Steven Zheng Swaroop Mishra ... Anush Mattapalli Ankur Taly Jingbo Shang Chen-Yu Lee Tomas Pfister RALM 75 31 0 11 Jul 2024
Scaling Retrieval-Based Language Models with a Trillion-Token Datastore Rulin Shao Jacqueline He Akari Asai Weijia Shi Tim Dettmers Sewon Min Luke Zettlemoyer Pang Wei Koh RALM 27 22 0 09 Jul 2024
VRSD: Rethinking Similarity and Diversity for Retrieval in Large Language Models Hang Gao Yongfeng Zhang 41 2 0 05 Jul 2024
Mixture of A Million Experts Xu Owen He MoE 31 25 0 04 Jul 2024
Do Generalised Classifiers really work on Human Drawn Sketches? Hmrishav Bandyopadhyay Pinaki Nath Chowdhury Aneeshan Sain Subhadeep Koley Tao Xiang A. Bhunia Yi-Zhe Song VLM 31 2 0 04 Jul 2024
DSLR: Document Refinement with Sentence-Level Re-ranking and Reconstruction to Enhance Retrieval-Augmented Generation Taeho Hwang Soyeong Jeong Sukmin Cho SeungYoon Han Jong C. Park RALM 30 1 0 04 Jul 2024
Contrast then Memorize: Semantic Neighbor Retrieval-Enhanced Inductive Multimodal Knowledge Graph Completion Yu Zhao Ying Zhang Baohang Zhou Xinying Qian Kehui Song Xiangrui Cai 29 2 0 03 Jul 2024
Efficient Nearest Neighbor based Uncertainty Estimation for Natural Language Processing Tasks Wataru Hashimoto Hidetaka Kamigaito Taro Watanabe 52 0 0 02 Jul 2024
SeaKR: Self-aware Knowledge Retrieval for Adaptive Retrieval Augmented Generation Zijun Yao Weijian Qi Liangming Pan S. Cao Linmei Hu Weichuan Liu Lei Hou Juanzi Li RALM 38 6 0 27 Jun 2024
RAVEN: Multitask Retrieval Augmented Vision-Language Learning Varun Nagaraj Rao Siddharth Choudhary Aditya Deshpande R. Satzoda Srikar Appalaraju RALM VLM 45 4 0 27 Jun 2024
Banishing LLM Hallucinations Requires Rethinking Generalization Johnny Li Saksham Consul Eda Zhou James Wong Naila Farooqui ... Zhuxiaona Wei Tian Wu Ben Echols Sharon Zhou Gregory Diamos LRM 20 10 0 25 Jun 2024
Mitigating Hallucination in Fictional Character Role-Play Nafis Sadeq Zhouhang Xie Byungkyu Kang Prarit Lamba Xiang Gao Julian McAuley HILM 38 7 0 25 Jun 2024
Ragnarök: A Reusable RAG Framework and Baselines for TREC 2024 Retrieval-Augmented Generation Track Ronak Pradeep Nandan Thakur Sahel Sharifymoghaddam Eric Zhang Ryan Nguyen Daniel Campos Nick Craswell Jimmy Lin 38 11 0 24 Jun 2024
Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization Cheng-Yu Hsieh Yung-Sung Chuang Chun-Liang Li Zifeng Wang Long T. Le ... James R. Glass Alexander Ratner Chen-Yu Lee Ranjay Krishna Tomas Pfister 40 30 0 23 Jun 2024
Memorizing Documents with Guidance in Large Language Models Bumjin Park Jaesik Choi KELM RALM 34 1 0 23 Jun 2024
GraphReader: Building Graph-based Agent to Enhance Long-Context Abilities of Large Language Models Shilong Li Yancheng He Hangyu Guo Xingyuan Bu Ge Bai ... Xingwei Qu Yangguang Li Wanli Ouyang Wenbo Su Bo Zheng RALM LLMAG 40 6 0 20 Jun 2024
InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationales Zhepei Wei Wei-Lin Chen Yu Meng RALM 53 12 0 19 Jun 2024
Evaluating $n$ -Gram Novelty of Language Models Using Rusty-DAWG William Merrill Noah A. Smith Yanai Elazar ELM TDI 38 9 0 18 Jun 2024
Think-then-Act: A Dual-Angle Evaluated Retrieval-Augmented Generation Yige Shen Hao Jiang Hua Qu Jihong Zhao RALM LRM 27 1 0 18 Jun 2024
PlanRAG: A Plan-then-Retrieval Augmented Generation for Generative Large Language Models as Decision Makers Myeonghwa Lee Seonho An Min-Soo Kim 3DV RALM 29 18 0 18 Jun 2024
Iterative Utility Judgment Framework via LLMs Inspired by Relevance in Philosophy Hengran Zhang Keping Bi Jiafeng Guo Xueqi Cheng 34 1 0 17 Jun 2024
Refiner: Restructure Retrieval Content Efficiently to Advance Question-Answering Capabilities Zhonghao Li Xuming Hu Aiwei Liu Kening Zheng S. Huang Hui Xiong RALM 115 8 0 17 Jun 2024
BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack Yuri Kuratov Aydar Bulatov Petr Anokhin Ivan Rodkin Dmitry Sorokin Artyom Sorokin Mikhail Burtsev RALM ALM LRM ReLM ELM 42 58 0 14 Jun 2024
Integrating Large Language Models with Graph-based Reasoning for Conversational Question Answering Parag Jain Mirella Lapata 31 0 0 14 Jun 2024
Multi-Modal Retrieval For Large Language Model Based Speech Recognition J. Kolehmainen Aditya Gourav Prashanth Gurunath Shivakumar Yile Gu Ankur Gandhe Ariya Rastrow Grant P. Strimel I. Bulyko 30 4 0 13 Jun 2024
Action2Sound: Ambient-Aware Generation of Action Sounds from Egocentric Videos Changan Chen Puyuan Peng Ami Baid Zihui Xue Wei-Ning Hsu David F. Harwath Kristen Grauman VGen 37 7 0 13 Jun 2024
Supportiveness-based Knowledge Rewriting for Retrieval-augmented Language Modeling Zile Qiao Wei Ye Yong Jiang Tong Mo Pengjun Xie Weiping Li Fei Huang Shikun Zhang KELM 25 4 0 12 Jun 2024
QuickLLaMA: Query-aware Inference Acceleration for Large Language Models Jingyao Li Han Shi Xin Jiang Zhenguo Li Hong Xu Jiaya Jia LRM 33 2 0 11 Jun 2024
Efficient k-Nearest-Neighbor Machine Translation with Dynamic Retrieval Yan Gao Zhiwei Cao Zhongjian Miao Baosong Yang Shiyu Liu Min Zhang Jinsong Su 37 0 0 10 Jun 2024
Retrieval & Fine-Tuning for In-Context Tabular Models Valentin Thomas Junwei Ma Rasa Hosseinzadeh Keyvan Golestan Guangwei Yu M. Volkovs Anthony L. Caterini 39 9 0 07 Jun 2024
Speaker-Smoothed kNN Speaker Adaptation for End-to-End ASR Shaojun Li Daimeng Wei Hengchao Shang Jiaxin Guo Zongyao Li Zhanglin Wu Zhiqiang Rao Yuanchang Luo Xianghui He Hao Yang 21 1 0 07 Jun 2024
MATTER: Memory-Augmented Transformer Using Heterogeneous Knowledge Sources Dongkyu Lee Chandana Satya Prakash Jack G. M. FitzGerald Jens Lehmann RALM 25 2 0 07 Jun 2024
Memorization in deep learning: A survey Jiaheng Wei Yanjun Zhang Leo Yu Zhang Ming Ding Chao Chen Kok-Leong Ong Jun Zhang Yang Xiang 42 6 0 06 Jun 2024
XL-HeadTags: Leveraging Multimodal Retrieval Augmentation for the Multilingual Generation of News Headlines and Tags Faisal Tareque Shohan Mir Tafseer Nayeem Samsul Islam Abu Ubaida Akash Shafiq R. Joty 34 2 0 06 Jun 2024
Extended Mind Transformers Phoebe Klett Thomas Ahle RALM 21 0 0 04 Jun 2024
CtrlA: Adaptive Retrieval-Augmented Generation via Probe-Guided Control Huanshuo Liu Hao Zhang Zhijiang Guo Kuicai Dong Xiangyang Li Yi Quan Lee Cong Zhang Yong-jin Liu 3DV 33 6 0 29 May 2024
Nearest Neighbor Speculative Decoding for LLM Generation and Attribution Minghan Li Xilun Chen Ari Holtzman Beidi Chen Jimmy Lin Wen-tau Yih Xi Victoria Lin RALM BDL 108 10 0 29 May 2024
Superposed Decoding: Multiple Generations from a Single Autoregressive Inference Pass Ethan Shen Alan Fan Sarah M Pratt Jae Sung Park Matthew Wallingford Sham Kakade Ari Holtzman Ranjay Krishna Ali Farhadi Aditya Kusupati 35 2 0 28 May 2024
Accelerating Inference of Retrieval-Augmented Generation via Sparse Context Selection Yun Zhu Jia-Chen Gu Caitlin Sikora Ho Ko Yinxiao Liu ... Lei Shu Liangchen Luo Lei Meng Bang Liu Jindong Chen RALM 22 14 0 25 May 2024
Mixture of In-Context Prompters for Tabular PFNs Derek Xu Olcay Cirit Reza Asadi Yizhou Sun Wei Wang 31 9 0 25 May 2024
Perception of Knowledge Boundary for Large Language Models through Semi-open-ended Question Answering Zhihua Wen Zhiliang Tian Z. Jian Zhen Huang Pei Ke Yifu Gao Minlie Huang Dongsheng Li 36 9 0 23 May 2024
xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token Xin Cheng Xun Wang Xingxing Zhang Tao Ge Si-Qing Chen Furu Wei Huishuai Zhang Dongyan Zhao 65 29 0 22 May 2024
Automated Evaluation of Retrieval-Augmented Language Models with Task-Specific Exam Generation Gauthier Guinet Behrooz Omidvar-Tehrani Anoop Deoras Laurent Callot RALM 65 17 0 22 May 2024
Towards Retrieval-Augmented Architectures for Image Captioning Sara Sarto Marcella Cornia Lorenzo Baraldi Alessandro Nicolosi Rita Cucchiara VLM 19 9 0 21 May 2024
ARAIDA: Analogical Reasoning-Augmented Interactive Data Annotation Chen Huang Yiping Jin Ilija Ilievski Wenqiang Lei Jiancheng Lv 35 3 0 20 May 2024
Feature-Adaptive and Data-Scalable In-Context Learning Jiahao Li Quan Wang L. Zhang Guoqing Jin Zhendong Mao 65 2 0 17 May 2024
Beyond Scaling Laws: Understanding Transformer Performance with Associative Memory Xueyan Niu Bo Bai Lei Deng Wei Han 31 6 0 14 May 2024
FlashBack:Efficient Retrieval-Augmented Language Modeling for Long Context Inference Runheng Liu Xingchen Xiao Heyan Huang Zewen Chi Zhijing Wu RALM KELM 18 0 0 07 May 2024
R4: Reinforced Retriever-Reorder-Responder for Retrieval-Augmented Large Language Models Taolin Zhang Dongyang Li Qizhou Chen Chengyu Wang Longtao Huang Hui Xue Xiaofeng He Junyuan Huang RALM KELM 45 0 0 04 May 2024