Retrieval as Attention: End-to-end Learning of Retrieval and Reading within a Single Transformer

5 December 2022

Graham Neubig

Papers citing "Retrieval as Attention: End-to-end Learning of Retrieval and Reading within a Single Transformer"

42 / 42 papers shown

Title
IterKey: Iterative Keyword Generation with LLMs for Enhanced Retrieval Augmented Generation Kazuki Hayashi Hidetaka Kamigaito Shinya Kouda Taro Watanabe RALM 29 0 0 13 May 2025
DioR: Adaptive Cognitive Detection and Contextual Retrieval Optimization for Dynamic Retrieval-Augmented Generation Hanghui Guo Jia Zhu Shimin Di Weijie Shi Zhangze Chen Jiajie Xu 30 0 0 14 Apr 2025
Attention Reveals More Than Tokens: Training-Free Long-Context Reasoning with Attention-guided Retrieval Yuwei Zhang Jayanth Srinivasa Gaowen Liu Jingbo Shang LRM LLMAG RALM 87 1 0 12 Mar 2025
Few-Shot Multilingual Open-Domain QA from 5 Examples Fan Jiang Tom Drummond Trevor Cohn 48 0 0 27 Feb 2025
You Only Use Reactive Attention Slice For Long Context Retrieval Yun Joon Soh Hanxian Huang Yuandong Tian Jishen Zhao RALM 33 0 0 03 Sep 2024
Enhancing and Accelerating Large Language Models via Instruction-Aware Contextual Compression Haowen Hou Fei Ma Binwen Bai Xinxin Zhu Fei Yu 28 0 0 28 Aug 2024
FastFiD: Improve Inference Efficiency of Open Domain Question Answering via Sentence Selection Yufei Huang Xu Han Maosong Sun 26 0 0 12 Aug 2024
Think-then-Act: A Dual-Angle Evaluated Retrieval-Augmented Generation Yige Shen Hao Jiang Hua Qu Jihong Zhao RALM LRM 27 1 0 18 Jun 2024
PlanRAG: A Plan-then-Retrieval Augmented Generation for Generative Large Language Models as Decision Makers Myeonghwa Lee Seonho An Min-Soo Kim 3DV RALM 29 18 0 18 Jun 2024
RE-RAG: Improving Open-Domain QA Performance and Interpretability with Relevance Estimator in Retrieval-Augmented Generation Kiseung Kim Jay-Yoon Lee RALM 40 5 0 09 Jun 2024
Chain of Agents: Large Language Models Collaborating on Long-Context Tasks Yusen Zhang Ruoxi Sun Yanfei Chen Tomas Pfister Rui Zhang Sercan Ö. Arik RALM AI4CE LLMAG 42 28 0 04 Jun 2024
Manipulating Large Language Models to Increase Product Visibility Aounon Kumar Himabindu Lakkaraju 33 7 0 11 Apr 2024
Towards Better Generalization in Open-Domain Question Answering by Mitigating Context Memorization Zixuan Zhang R. Reddy Kevin Small Tong Zhang Heng Ji 32 1 0 02 Apr 2024
DRAGIN: Dynamic Retrieval Augmented Generation based on the Information Needs of Large Language Models Weihang Su Yichen Tang Qingyao Ai Zhijing Wu Yiqun Liu 3DV RALM AI4TS SyDa 51 18 0 15 Mar 2024
Unsupervised Real-Time Hallucination Detection based on the Internal States of Large Language Models Weihang Su Changyue Wang Qingyao Ai Hu Yiran Zhijing Wu Yujia Zhou Yiqun Liu HILM 37 28 0 11 Mar 2024
RNNs are not Transformers (Yet): The Key Bottleneck on In-context Retrieval Kaiyue Wen Xingyu Dang Kaifeng Lyu 44 24 0 28 Feb 2024
Pre-training Cross-lingual Open Domain Question Answering with Large-scale Synthetic Supervision Fan Jiang Tom Drummond Trevor Cohn CLIP ELM LRM 26 3 0 26 Feb 2024
Instruction-tuned Language Models are Better Knowledge Learners Zhengbao Jiang Zhiqing Sun Weijia Shi Pedro Rodriguez Chunting Zhou Graham Neubig Xi Victoria Lin Wen-tau Yih Srinivasan Iyer KELM 38 33 0 20 Feb 2024
Metacognitive Retrieval-Augmented Large Language Models Yujia Zhou Zheng Liu Jiajie Jin Jian-yun Nie Zhicheng Dou RALM KELM AIFin LRM 27 14 0 18 Feb 2024
An Enhanced Prompt-Based LLM Reasoning Scheme via Knowledge Graph-Integrated Collaboration Yihao Li Ru Zhang Jianyi Liu LRM 55 16 0 07 Feb 2024
The What, Why, and How of Context Length Extension Techniques in Large Language Models -- A Detailed Survey Saurav Pawar S.M. Towhidul Islam Tonmoy S. M. M. Zaman Vinija Jain Aman Chadha Amitava Das 29 27 0 15 Jan 2024
Large Language Models for Social Networks: Applications, Challenges, and Solutions Jingying Zeng Richard Huang Waleed Malik Langxuan Yin Bojan Babic Danny Shacham Xiao Yan Jaewon Yang Qi He 22 3 0 04 Jan 2024
Evaluating Generative Ad Hoc Information Retrieval Lukas Gienapp Harrisen Scells Niklas Deckers Janek Bevendorff Shuai Wang ... Maik Frobe Guide Zucoon Benno Stein Matthias Hagen Martin Potthast RALM 37 11 0 08 Nov 2023
1-PAGER: One Pass Answer Generation and Evidence Retrieval Palak Jain Livio Baldini Soares Tom Kwiatkowski RALM 20 6 0 25 Oct 2023
How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advances Zihan Zhang Meng Fang Lingxi Chen Mohammad-Reza Namazi-Rad Jun Wang KELM 19 21 0 11 Oct 2023
Retrieval meets Long Context Large Language Models Peng-Tao Xu Wei Ping Xianchao Wu Lawrence C. McAfee Chen Zhu Zihan Liu Sandeep Subramanian Evelina Bakhturina M. Shoeybi Bryan Catanzaro RALM LRM 14 79 0 04 Oct 2023
LeanDojo: Theorem Proving with Retrieval-Augmented Language Models Kaiyu Yang Aidan M. Swope Alex Gu Rahul Chalamala Peiyang Song Shixing Yu Saad Godil R. Prenger Anima Anandkumar RALM 19 207 0 27 Jun 2023
Extending Context Window of Large Language Models via Positional Interpolation Shouyuan Chen Sherman Wong Liangjian Chen Yuandong Tian 12 494 0 27 Jun 2023
Long-range Language Modeling with Self-retrieval Ohad Rubin Jonathan Berant RALM KELM 19 18 0 23 Jun 2023
GLIMMER: generalized late-interaction memory reranker Michiel de Jong Yury Zemlyanskiy Nicholas FitzGerald Sumit Sanghai William W. Cohen Joshua Ainslie RALM 41 4 0 17 Jun 2023
Robust Natural Language Understanding with Residual Attention Debiasing Fei Wang James Y. Huang Tianyi Yan Wenxuan Zhou Muhao Chen 29 10 0 28 May 2023
Landmark Attention: Random-Access Infinite Context Length for Transformers Amirkeivan Mohtashami Martin Jaggi LLMAG 19 149 0 25 May 2023
Query Rewriting for Retrieval-Augmented Large Language Models Xinbei Ma Yeyun Gong Pengcheng He Hai Zhao Nan Duan KELM LRM 27 101 0 23 May 2023
ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval Yue Yu Yuchen Zhuang Rongzhi Zhang Yu Meng Jiaming Shen Chao Zhang VLM 30 33 0 18 May 2023
Active Retrieval Augmented Generation Zhengbao Jiang Frank F. Xu Luyu Gao Zhiqing Sun Qian Liu Jane Dwivedi-Yu Yiming Yang Jamie Callan Graham Neubig RALM 9 252 0 11 May 2023
BRENT: Bidirectional Retrieval Enhanced Norwegian Transformer Lucas Georges Gabriel Charpentier Sondre Wold David Samuel Egil Rønningstad KELM RALM 9 2 0 19 Apr 2023
Training Language Models with Memory Augmentation Zexuan Zhong Tao Lei Danqi Chen RALM 232 127 0 25 May 2022
PLAID: An Efficient Engine for Late Interaction Retrieval Keshav Santhanam Omar Khattab Christopher Potts Matei A. Zaharia VLM 58 72 0 19 May 2022
Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval Luyu Gao Jamie Callan RALM 157 329 0 12 Aug 2021
BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models Nandan Thakur Nils Reimers Andreas Rucklé Abhishek Srivastava Iryna Gurevych VLM 229 966 0 17 Apr 2021
Distilling Knowledge from Reader to Retriever for Question Answering Gautier Izacard Edouard Grave RALM 180 251 0 08 Dec 2020
RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering Yingqi Qu Yuchen Ding Jing Liu Kai Liu Ruiyang Ren Xin Zhao Daxiang Dong Hua-Hong Wu Haifeng Wang RALM OffRL 214 593 0 16 Oct 2020