InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory

7 February 2024

Chaojun Xiao

Pengle Zhang

Xu Han

Guangxuan Xiao

Yankai Lin

Zhengyan Zhang

Zhiyuan Liu

Maosong Sun

LLMAG

ArXiv PDF HTML

Papers citing "InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory"

29 / 29 papers shown

Title
RetroInfer: A Vector-Storage Approach for Scalable Long-Context LLM Inference Y. Chen J. Zhang Baotong Lu Qianxi Zhang Chengruidong Zhang ... Chen Chen Mingxing Zhang Yuqing Yang Fan Yang Mao Yang 32 0 0 05 May 2025
Memorization and Knowledge Injection in Gated LLMs Xu Pan Ely Hahami Zechen Zhang H. Sompolinsky KELM CLL RALM 104 0 0 30 Apr 2025
The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs Piotr Nawrot Robert Li Renjie Huang Sebastian Ruder Kelly Marchisio E. Ponti 25 0 0 24 Apr 2025
AlayaDB: The Data Foundation for Efficient and Effective Long-context LLM Inference Yangshen Deng Zhengxin You Long Xiang Qilong Li Peiqi Yuan ... Man Lung Yiu Huan Li Qiaomu Shen Rui Mao Bo Tang 31 0 0 14 Apr 2025
Harnessing the Unseen: The Hidden Influence of Intrinsic Knowledge in Long-Context Language Models Yu Fu Haz Sameen Shahgir Hui Liu Xianfeng Tang Qi He Yue Dong KELM 44 0 0 11 Apr 2025
Adaptive Computation Pruning for the Forgetting Transformer Zhixuan Lin J. Obando-Ceron Xu Owen He Aaron C. Courville 30 0 0 09 Apr 2025
XAttention: Block Sparse Attention with Antidiagonal Scoring Ruyi Xu Guangxuan Xiao Haofeng Huang Junxian Guo Song Han 64 3 0 20 Mar 2025
LLMs Know What to Drop: Self-Attention Guided KV Cache Eviction for Efficient Long-Context Inference G. Wang Shubhangi Upasani Chen Henry Wu Darshan Gandhi Jonathan Li Changran Hu Bo Li Urmish Thakker 75 0 0 11 Mar 2025
Predicting Team Performance from Communications in Simulated Search-and-Rescue Ali Jalal-Kamali Nikolos Gurney David Pynadath AI4TS 102 8 0 05 Mar 2025
Streaming Video Question-Answering with In-context Video KV-Cache Retrieval Shangzhe Di Zhelun Yu Guanghao Zhang Haoyuan Li Tao Zhong Hao Cheng Bolin Li Wanggui He Fangxun Shu Hao Jiang 53 4 0 01 Mar 2025
WildLong: Synthesizing Realistic Long-Context Instruction Data at Scale Jiaxi Li Xingxing Zhang Xun Wang Xiaolong Huang Li Dong Liang Wang Si-Qing Chen Wei Lu Furu Wei SyDa 60 0 0 23 Feb 2025
InfiniteHiP: Extending Language Model Context Up to 3 Million Tokens on a Single GPU Heejun Lee G. Park Jaduk Suh Sung Ju Hwang 82 1 0 13 Feb 2025
SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantization Jintao Zhang Haofeng Huang Pengle Zhang Jia wei Jun-Jie Zhu Jianfei Chen VLM MQ 50 2 0 17 Nov 2024
LongSafety: Enhance Safety for Long-Context LLMs Mianqiu Huang Xiaoran Liu Shaojun Zhou Mozhi Zhang Chenkun Tan ... Zhikai Lei Linlin Li Q. Liu Yaqian Zhou Xipeng Qiu ELM ALM 32 0 0 11 Nov 2024
TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection Wei Yu Wu Zhuoshi Pan Chao Wang L. Chen Y. Bai Kun Fu Z. Wang Hui Xiong Hui Xiong LLMAG 31 5 0 05 Nov 2024
Emotional RAG: Enhancing Role-Playing Agents through Emotional Retrieval Le Huang Hengzhi Lan Zijun Sun Chuan Shi Ting Bai 46 0 0 30 Oct 2024
$LLM$\times$MapReduce: Simplified Long-Sequence Processing using Large Language Models$ LLM $\times$ MapReduce: Simplified Long-Sequence Processing using Large Language Models Zihan Zhou C. Li Xinyi Chen Shuo Wang Yu Chao ... Zhixing Tan Xu Han Xiaodong Shi Zhiyuan Liu Maosong Sun 24 0 0 12 Oct 2024
On the token distance modeling ability of higher RoPE attention dimension Xiangyu Hong Che Jiang Biqing Qi Fandong Meng Mo Yu Bowen Zhou Jie Zhou 20 1 0 11 Oct 2024
Correlation-Aware Select and Merge Attention for Efficient Fine-Tuning and Context Length Extension Ning Wang Zekun Li Tongxin Bai Guoqi Li 22 0 0 05 Oct 2024
SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration Jintao Zhang Jia wei Pengle Zhang Jun-Jie Zhu Jun Zhu Jianfei Chen VLM MQ 74 18 0 03 Oct 2024
Schrodinger's Memory: Large Language Models Wei Wang Qing Li 19 1 0 16 Sep 2024
The CAP Principle for LLM Serving: A Survey of Long-Context Large Language Model Serving Pai Zeng Zhenyu Ning Jieru Zhao Weihao Cui Mengwei Xu Liwei Guo Xusheng Chen Yizhou Shan LLMAG 35 4 0 18 May 2024
SnapKV: LLM Knows What You are Looking for Before Generation Yuhong Li Yingbing Huang Bowen Yang Bharat Venkitesh Acyr F. Locatelli Hanchen Ye Tianle Cai Patrick Lewis Deming Chen VLM 75 148 0 22 Apr 2024
Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention Tsendsuren Munkhdalai Manaal Faruqui Siddharth Gopal LRM LLMAG CLL 79 101 0 10 Apr 2024
Unlimiformer: Long-Range Transformers with Unlimited Length Input Amanda Bertsch Uri Alon Graham Neubig Matthew R. Gormley RALM 94 122 0 02 May 2023
Generative Agents: Interactive Simulacra of Human Behavior J. Park Joseph C. O'Brien Carrie J. Cai Meredith Ringel Morris Percy Liang Michael S. Bernstein LM&Ro AI4CE 209 1,701 0 07 Apr 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 234 690 0 27 Aug 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 249 1,982 0 28 Jul 2020