Beyond the Limits: A Survey of Techniques to Extend the Context Length
in Large Language Models

Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models

3 February 2024

Mehdi Rezagholizadeh

Papers citing "Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models"

11 / 11 papers shown

Title
Attention Condensation via Sparsity Induced Regularized Training Eli Sason Darya Frolova Boris Nazarov Felix Goldberd 72 0 0 03 Mar 2025
Wormhole Memory: A Rubik's Cube for Cross-Dialogue Retrieval Libo Wang 45 0 0 24 Jan 2025
Towards LifeSpan Cognitive Systems Yu Wang Chi Han Tongtong Wu Xiaoxin He Wangchunshu Zhou ... Zexue He Wei Wang Gholamreza Haffari Heng Ji Julian McAuley KELM CLL 81 1 0 20 Sep 2024
Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost Sania Nayab Giulio Rossolini Giorgio Buttazzo Nicolamaria Manes F. Giacomelli Nicolamaria Manes Fabrizio Giacomelli LRM 32 22 0 29 Jul 2024
LLM in a flash: Efficient Large Language Model Inference with Limited Memory Keivan Alizadeh-Vahid Iman Mirzadeh Dmitry Belenko Karen Khatamifard Minsik Cho C. C. D. Mundo Mohammad Rastegari Mehrdad Farajtabar 68 104 0 12 Dec 2023
M4LE: A Multi-Ability Multi-Range Multi-Task Multi-Domain Long-Context Evaluation Benchmark for Large Language Models Wai-Chung Kwan Xingshan Zeng Yufei Wang Yusen Sun Liangyou Li Lifeng Shang Qun Liu Kam-Fai Wong ELM 89 10 0 30 Oct 2023
LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression Huiqiang Jiang Qianhui Wu Xufang Luo Dongsheng Li Chin-Yew Lin Yuqing Yang Lili Qiu RALM 96 179 0 10 Oct 2023
Resurrecting Recurrent Neural Networks for Long Sequences Antonio Orvieto Samuel L. Smith Albert Gu Anushan Fernando Çağlar Gülçehre Razvan Pascanu Soham De 83 258 0 11 Mar 2023
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 234 690 0 27 Aug 2021
A Simple and Effective Positional Encoding for Transformers Pu-Chin Chen Henry Tsai Srinadh Bhojanapalli Hyung Won Chung Yin-Wen Chang Chun-Sung Ferng 33 61 0 18 Apr 2021
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 228 502 0 12 Mar 2020