Challenges in Deploying Long-Context Transformers: A Theoretical Peak Performance Analysis

14 May 2024

Papers citing "Challenges in Deploying Long-Context Transformers: A Theoretical Peak Performance Analysis"

16 / 16 papers shown

Title
The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs Piotr Nawrot Robert Li Renjie Huang Sebastian Ruder Kelly Marchisio E. Ponti 25 0 0 24 Apr 2025
An LMM for Efficient Video Understanding via Reinforced Compression of Video Cubes Ji Qi Y. Yao Yushi Bai Bin Xu Juanzi Li Zhiyuan Liu Tat-Seng Chua 29 0 0 21 Apr 2025
SWAN-GPT: An Efficient and Scalable Approach for Long-Context Language Modeling Krishna C. Puvvada Faisal Ladhak Santiago Akle Serrano Cheng-Ping Hsieh Shantanu Acharya ... Fei Jia Samuel Kriman Simeng Sun Dima Rekesh Boris Ginsburg RALM 54 0 0 11 Apr 2025
FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference Xunhao Lai Jianqiao Lu Yao Luo Yiyuan Ma Xun Zhou 63 5 0 28 Feb 2025
freePruner: A Training-free Approach for Large Multimodal Model Acceleration Bingxin Xu Yuzhang Shang Yunhao Ge Qian Lou Yan Yan 94 3 0 23 Nov 2024
StreamAdapter: Efficient Test Time Adaptation from Contextual Streams Dilxat Muhtar Yelong Shen Y. Yang Xiaodong Liu Yadong Lu ... Feng Sun Xueliang Zhang Jianfeng Gao Weizhu Chen Qi Zhang TTA 59 0 0 14 Nov 2024
Residual vector quantization for KV cache compression in large language model Ankur Kumar MQ 24 0 0 21 Oct 2024
Geometric Collaborative Filtering with Convergence Hisham Husain Julien Monteil FedML 23 5 0 04 Oct 2024
Analog In-Memory Computing Attention Mechanism for Fast and Energy-Efficient Large Language Models Nathan Leroux Paul-Philipp Manea Chirag Sudarshan Jan Finkbeiner Sebastian Siegel J. Strachan Emre Neftci 13 1 0 28 Sep 2024
Interpolating Video-LLMs: Toward Longer-sequence LMMs in a Training-free Manner Yuzhang Shang Bingxin Xu Weitai Kang Mu Cai Yuheng Li Zehao Wen Zhen Dong Kurt Keutzer Yong Jae Lee Yan Yan 33 0 0 19 Sep 2024
Let the Code LLM Edit Itself When You Edit the Code Zhenyu He Jun Zhang Shengjie Luo Jingjing Xu Z. Zhang Di He KELM 29 0 0 03 Jul 2024
MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention Huiqiang Jiang Yucheng Li Chengruidong Zhang Qianhui Wu Xufang Luo ... Amir H. Abdi Dongsheng Li Chin-Yew Lin Yuqing Yang L. Qiu 65 1 0 02 Jul 2024
Block Transformer: Global-to-Local Language Modeling for Fast Inference Namgyu Ho Sangmin Bae Taehyeon Kim Hyunjik Jo Yireun Kim Tal Schuster Adam Fisch James Thorne Se-Young Yun 45 6 0 04 Jun 2024
HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models Bernal Jiménez Gutiérrez Yiheng Shu Yu Gu Michihiro Yasunaga Yu-Chuan Su RALM CLL 48 27 0 23 May 2024
SnapKV: LLM Knows What You are Looking for Before Generation Yuhong Li Yingbing Huang Bowen Yang Bharat Venkitesh Acyr F. Locatelli Hanchen Ye Tianle Cai Patrick Lewis Deming Chen VLM 75 148 0 22 Apr 2024
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 220 3,054 0 23 Jan 2020