Lost in the Middle: How Language Models Use Long Contexts

6 July 2023

Papers citing "Lost in the Middle: How Language Models Use Long Contexts"

50 / 178 papers shown

Title
By My Eyes: Grounding Multimodal Large Language Models with Sensor Data via Visual Prompting Hyungjun Yoon Biniyam Aschalew Tolera Taesik Gong Kimin Lee Sung-Ju Lee 28 6 0 15 Jul 2024
Cohesive Conversations: Enhancing Authenticity in Multi-Agent Simulated Dialogues Kuanchao Chu Yi-Pei Chen Hideki Nakayama LLMAG 34 2 0 13 Jul 2024
SPIQA: A Dataset for Multimodal Question Answering on Scientific Papers Shraman Pramanick Rama Chellappa Subhashini Venugopalan 43 13 0 12 Jul 2024
SpreadsheetLLM: Encoding Spreadsheets for Large Language Models Yuzhang Tian Jianbo Zhao Haoyu Dong Junyu Xiong Shiyu Xia ... Yun Lin José Cambronero Yeye He Shi Han Dongmei Zhang LMTD 36 6 0 12 Jul 2024
Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting Zilong Wang Zifeng Wang Long Le Huaixiu Steven Zheng Swaroop Mishra ... Anush Mattapalli Ankur Taly Jingbo Shang Chen-Yu Lee Tomas Pfister RALM 70 30 0 11 Jul 2024
FsPONER: Few-shot Prompt Optimization for Named Entity Recognition in Domain-specific Scenarios Yongjian Tang Rakebul Hasan Thomas Runkler 58 2 0 10 Jul 2024
ConCodeEval: Evaluating Large Language Models for Code Constraints in Domain-Specific Languages Mehant Kammakomati Sameer Pimparkhede Srikanth G. Tamilselvam Prince Kumar Pushpak Bhattacharyya ALM 35 0 0 03 Jul 2024
Eliminating Position Bias of Language Models: A Mechanistic Approach Ziqi Wang Hanlin Zhang Xiner Li Kuan-Hao Huang Chi Han Shuiwang Ji Sham Kakade Hao Peng Heng Ji 49 12 0 01 Jul 2024
Entropy-Based Decoding for Retrieval-Augmented Large Language Models Zexuan Qiu Zijing Ou Bin Wu Jingjing Li Aiwei Liu Irwin King KELM RALM 25 5 0 25 Jun 2024
On the Transformations across Reward Model, Parameter Update, and In-Context Prompt Deng Cai Huayang Li Tingchen Fu Siheng Li Weiwen Xu ... Leyang Cui Yan Wang Lemao Liu Taro Watanabe Shuming Shi KELM 26 2 0 24 Jun 2024
SEAM: A Stochastic Benchmark for Multi-Document Tasks Gili Lior Avi Caciularu Arie Cattan Shahar Levy Ori Shapira Gabriel Stanovsky RALM 33 4 0 23 Jun 2024
First Heuristic Then Rational: Dynamic Use of Heuristics in Language Model Reasoning Yoichi Aoki Keito Kudo Tatsuki Kuribayashi Shusaku Sone Masaya Taniguchi Keisuke Sakaguchi Kentaro Inui LRM 19 1 0 23 Jun 2024
A Tale of Trust and Accuracy: Base vs. Instruct LLMs in RAG Systems Florin Cuconasu Giovanni Trappolini Nicola Tonellotto Fabrizio Silvestri 49 2 0 21 Jun 2024
Insights into LLM Long-Context Failures: When Transformers Know but Don't Tell Taiming Lu Muhan Gao Kuai Yu Adam Byerly Daniel Khashabi 37 11 0 20 Jun 2024
SEC-QA: A Systematic Evaluation Corpus for Financial QA Viet Dac Lai Michael Krumdick Charles Lovering Varshini Reddy Craig W. Schmidt Chris Tanner 41 3 0 20 Jun 2024
DeciMamba: Exploring the Length Extrapolation Potential of Mamba Assaf Ben-Kish Itamar Zimerman Shady Abu Hussein Nadav Cohen Amir Globerson Lior Wolf Raja Giryes Mamba 61 12 0 20 Jun 2024
MDCR: A Dataset for Multi-Document Conditional Reasoning Peter Baile Chen Yi Zhang Chunwei Liu Sejal Gupta Yoon Kim Michael Cafarella 39 2 0 17 Jun 2024
R-Eval: A Unified Toolkit for Evaluating Domain Knowledge of Retrieval Augmented Large Language Models Shangqing Tu Yuanchun Wang Jifan Yu Yuyang Xie Yaran Shi Xiaozhi Wang Jing Zhang Lei Hou Juanzi Li ELM 27 3 0 17 Jun 2024
TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategy Yiqun Chen Qi Liu Yi Zhang Weiwei Sun Daiting Shi Jiaxin Mao Dawei Yin Jiaxin Mao Dawei Yin 33 5 0 17 Jun 2024
Large Language Models for Automatic Milestone Detection in Group Discussions Zhuoxu Duan Zhengye Yang Samuel Westby Christoph Riedl B. F. Welles Richard J. Radke 21 0 0 16 Jun 2024
Applications of Generative AI in Healthcare: algorithmic, ethical, legal and societal considerations Onyekachukwu R. Okonji Kamol Yunusov Bonnie Gordon MedIm 25 3 0 15 Jun 2024
Talking Heads: Understanding Inter-layer Communication in Transformer Language Models Jack Merullo Carsten Eickhoff Ellie Pavlick 47 12 0 13 Jun 2024
The Impact of Quantization on Retrieval-Augmented Generation: An Analysis of Small LLMs Mert Yazan Suzan Verberne F. Situmeang MQ 26 3 0 10 Jun 2024
Chain of Agents: Large Language Models Collaborating on Long-Context Tasks Yusen Zhang Ruoxi Sun Yanfei Chen Tomas Pfister Rui Zhang Sercan Ö. Arik RALM AI4CE LLMAG 42 28 0 04 Jun 2024
Mitigate Position Bias in Large Language Models via Scaling a Single Dimension Yijiong Yu Huiqiang Jiang Xufang Luo Qianhui Wu Chin-Yew Lin Dongsheng Li Yuqing Yang Yongfeng Huang L. Qiu 35 9 0 04 Jun 2024
Analyzing Temporal Complex Events with Large Language Models? A Benchmark towards Temporal, Long Context Understanding Zhihan Zhang Yixin Cao Chenchen Ye Yunshan Ma Lizi Liao Tat-Seng Chua 21 9 0 04 Jun 2024
Unraveling and Mitigating Retriever Inconsistencies in Retrieval-Augmented Large Language Models Mingda Li Xinyu Li Yifan Chen Wenfeng Xuan Weinan Zhang RALM 29 2 0 31 May 2024
One Token Can Help! Learning Scalable and Pluggable Virtual Tokens for Retrieval-Augmented Large Language Models Yutao Zhu Zhaoheng Huang Zhicheng Dou Ji-Rong Wen RALM 37 5 0 30 May 2024
CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion Jiayi Yao Hanchen Li Yuhan Liu Siddhant Ray Yihua Cheng Qizheng Zhang Kuntai Du Shan Lu Junchen Jiang 42 12 0 26 May 2024
Towards Completeness-Oriented Tool Retrieval for Large Language Models Changle Qu Sunhao Dai Xiaochi Wei Hengyi Cai Shuaiqiang Wang Dawei Yin Jun Xu Jirong Wen KELM 22 7 0 25 May 2024
Imagery as Inquiry: Exploring A Multimodal Dataset for Conversational Recommendation Se-eun Yoon Hyunsik Jeon Julian McAuley 35 0 0 23 May 2024
Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models Zhangyue Yin Qiushi Sun Qipeng Guo Zhiyuan Zeng Xiaonan Li ... Qinyuan Cheng Ding Wang Xiaofeng Mou Xipeng Qiu XuanJing Huang LRM 41 3 0 21 May 2024
KG-RAG: Bridging the Gap Between Knowledge and Creativity Diego Sanmartin RALM 34 35 0 20 May 2024
DuetSim: Building User Simulator with Dual Large Language Models for Task-Oriented Dialogues Xiang Luo Zhiwen Tang Jin Wang Xuejie Zhang 21 4 0 16 May 2024
In-Context Learning with Long-Context Models: An In-Depth Exploration Amanda Bertsch Maor Ivgi Uri Alon Jonathan Berant Matthew R. Gormley Matthew R. Gormley Graham Neubig ReLM AIMat 81 64 0 30 Apr 2024
From Matching to Generation: A Survey on Generative Information Retrieval Xiaoxi Li Jiajie Jin Yujia Zhou Yuyao Zhang Peitian Zhang Yutao Zhu Zhicheng Dou 3DV 62 36 0 23 Apr 2024
RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation Chao Jin Zili Zhang Xuanlin Jiang Fangyue Liu Xin Liu Xuanzhe Liu Xin Jin 32 36 0 18 Apr 2024
MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memory Ali Modarressi Abdullatif Köksal Ayyoob Imani Mohsen Fayyaz Hinrich Schütze KELM 93 9 0 17 Apr 2024
ChatShop: Interactive Information Seeking with Language Agents Sanxing Chen Sam Wiseman Bhuwan Dhingra KELM 18 7 0 15 Apr 2024
LLM In-Context Recall is Prompt Dependent Daniel Machlab Rick Battle LLMAG 16 12 0 13 Apr 2024
AdapterSwap: Continuous Training of LLMs with Data Removal and Access-Control Guarantees William Fleshman Aleem Khan Marc Marone Benjamin Van Durme CLL KELM 42 3 0 12 Apr 2024
ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models Jinheon Baek S. Jauhar Silviu Cucerzan Sung Ju Hwang AI4CE LLMAG LM&Ro 34 36 0 11 Apr 2024
Can LLMs Separate Instructions From Data? And What Do We Even Mean By That? Egor Zverev Sahar Abdelnabi Soroush Tabesh Mario Fritz Christoph H. Lampert 41 19 0 11 Mar 2024
A Modular Approach for Multimodal Summarization of TV Shows Louis Mahon Mirella Lapata 21 9 0 06 Mar 2024
RouteExplainer: An Explanation Framework for Vehicle Routing Problem Daisuke Kikuta Hiroki Ikeuchi Kengo Tajiri Yuusuke Nakano 21 1 0 06 Mar 2024
The First Place Solution of WSDM Cup 2024: Leveraging Large Language Models for Conversational Multi-Doc QA Yiming Li Zhao Zhang 16 1 0 28 Feb 2024
Analysing The Impact of Sequence Composition on Language Model Pre-Training Yu Zhao Yuanbin Qu Konrad Staniszewski Szymon Tworkowski Wei Liu Piotr Milo's Yuxiang Wu Pasquale Minervini 24 13 0 21 Feb 2024
User-LLM: Efficient LLM Contextualization with User Embeddings Lin Ning Luyang Liu Jiaxing Wu Neo Wu D. Berlowitz Sushant Prakash Bradley Green S. O’Banion Jun Xie 37 32 0 21 Feb 2024
Where is the answer? Investigating Positional Bias in Language Model Knowledge Extraction Kuniaki Saito Kihyuk Sohn Chen-Yu Lee Yoshitaka Ushiku 48 2 0 16 Feb 2024
On the Efficacy of Eviction Policy for Key-Value Constrained Generative Language Model Inference Siyu Ren Kenny Q. Zhu 13 27 0 09 Feb 2024