Title
Sparse Attention Remapping with Clustering for Efficient LLM Decoding on PIM Zehao Fan Garrett Gagnon Zhenyu Liu Liu Liu 19 0 0 09 May 2025
R-Sparse: Rank-Aware Activation Sparsity for Efficient LLM Inference Zhenyu (Allen) Zhang Zechun Liu Yuandong Tian Harshit Khaitan Z. Wang Steven Li 54 0 0 28 Apr 2025
TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate A. Zandieh Majid Daliri Majid Hadian Vahab Mirrokni MQ 74 0 0 28 Apr 2025
An Empirical Study on Prompt Compression for Large Language Models Z. Zhang Jinyi Li Yihuai Lan X. Wang Hao Wang MQ 42 0 0 24 Apr 2025
Random Long-Context Access for Mamba via Hardware-aligned Hierarchical Sparse Attention Xiang Hu Jiaqi Leng Jun Zhao Kewei Tu Wei Wu Mamba 45 0 0 23 Apr 2025
PARD: Accelerating LLM Inference with Low-Cost PARallel Draft Model Adaptation Zihao An Huajun Bai Z. Liu Dong Li E. Barsoum 54 0 0 23 Apr 2025
From Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMs Yaxiong Wu Sheng Liang Chen Zhang Y. Wang Y. Zhang Huifeng Guo Ruiming Tang Y. Liu KELM 36 0 0 22 Apr 2025
KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environments Junyoung Park Dalton Jones Matt Morse Raghavv Goel Mingu Lee Chris Lott 22 0 0 21 Apr 2025
Efficient Pretraining Length Scaling Bohong Wu Shen Yan Sijun Zhang Jianqiao Lu Yutao Zeng Ya Wang Xun Zhou 39 0 0 21 Apr 2025
KeepKV: Eliminating Output Perturbation in KV Cache Compression for Efficient LLMs Inference Yuxuan Tian Zihan Wang Yebo Peng Aomufei Yuan Z. Wang Bairen Yi Xin Liu Yong Cui Tong Yang 29 0 0 14 Apr 2025
HELIOS: Adaptive Model And Early-Exit Selection for Efficient LLM Inference Serving Avinash Kumar Shashank Nag Jason Clemons L. John Poulami Das 24 0 0 14 Apr 2025
Adaptive Computation Pruning for the Forgetting Transformer Zhixuan Lin J. Obando-Ceron Xu Owen He Aaron C. Courville 30 0 0 09 Apr 2025
Hogwild! Inference: Parallel LLM Generation via Concurrent Attention Gleb Rodionov Roman Garipov Alina Shutova George Yakushev Vage Egiazarian Anton Sinitsin Denis Kuznedelev Dan Alistarh LRM 27 1 0 08 Apr 2025
LagKV: Lag-Relative Information of the KV Cache Tells Which Tokens Are Important Manlai Liang JiaMing Zhang Xiong Li Jinlong Li MQ 33 0 0 07 Apr 2025
Saliency-driven Dynamic Token Pruning for Large Language Models Yao Tao Yehui Tang Yun Wang Mingjian Zhu Hailin Hu Yunhe Wang 32 0 0 06 Apr 2025
Cognitive Memory in Large Language Models Lianlei Shan Shixian Luo Zezhou Zhu Yu Yuan Yong Wu LLMAG KELM 63 1 0 03 Apr 2025
SentenceKV: Efficient LLM Inference via Sentence-Level Semantic KV Caching Yuxuan Zhu Ali Falahati David H. Yang Mohammad Mohammadi Amiri 54 0 0 01 Apr 2025
Efficient LLaMA-3.2-Vision by Trimming Cross-attended Visual Features Jewon Lee Ki-Ung Song Seungmin Yang Donguk Lim Jaeyeon Kim Wooksu Shin Bo-Kyeong Kim Yong Jae Lee Tae-Ho Kim VLM 51 0 0 01 Apr 2025
AirCache: Activating Inter-modal Relevancy KV Cache Compression for Efficient Large Vision-Language Model Inference Kai Huang Hao Zou Bochen Wang Ye Xi Zhen Xie Hao Wang VLM 42 0 0 31 Mar 2025
SQuat: Subspace-orthogonal KV Cache Quantization Hao Wang Ligong Han Kai Xu Akash Srivastava MQ 43 0 0 31 Mar 2025
FastVAR: Linear Visual Autoregressive Modeling via Cached Token Pruning Hang Guo Yawei Li Taolin Zhang J. Wang Tao Dai Shu-Tao Xia Luca Benini 60 1 0 30 Mar 2025
Cocktail: Chunk-Adaptive Mixed-Precision Quantization for Long-Context LLM Inference Wei Tao Bin Zhang Xiaoyang Qu Jiguang Wan Jianzong Wang 31 1 0 30 Mar 2025
Skip-Vision: Efficient and Scalable Acceleration of Vision-Language Models via Adaptive Token Skipping Weili Zeng Ziyuan Huang Kaixiang Ji Yichao Yan VLM 42 1 0 26 Mar 2025
LogQuant: Log-Distributed 2-Bit Quantization of KV Cache with Superior Accuracy Preservation Han Chen Zicong Jiang Zining Zhang Bingsheng He Pingyi Luo M. Lu Yuqiang Chen MQ 42 0 0 25 Mar 2025
TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model Cheng Yang Yang Sui Jinqi Xiao Lingyi Huang Yu Gong ... Jinghua Yan Y. Bai P. Sadayappan Xia Hu Bo Yuan VLM 53 0 0 24 Mar 2025
Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models Yang Sui Yu-Neng Chuang Guanchu Wang Jiamu Zhang Tianyi Zhang ... Hongyi Liu Andrew Wen Shaochen Zhong Hanjie Chen OffRL ReLM LRM 62 21 0 20 Mar 2025
SpeCache: Speculative Key-Value Caching for Efficient Generation of LLMs Shibo Jie Yehui Tang Kai Han Zhi-Hong Deng Jing Han 87 0 0 20 Mar 2025
Plug-and-Play 1.x-Bit KV Cache Quantization for Video Large Language Models Keda Tao Haoxuan You Yang Sui Can Qin H. Wang VLM MQ 84 0 0 20 Mar 2025
Growing a Twig to Accelerate Large Vision-Language Models Zhenwei Shao Mingyang Wang Zhou Yu Wenwen Pan Yan Yang Tao Wei H. Zhang Ning Mao Wei Chen Jun Yu VLM 59 1 0 18 Mar 2025
LazyMAR: Accelerating Masked Autoregressive Models via Feature Caching Feihong Yan Qingyan Wei Jiayi Tang J. Li Y. Wang Xuming Hu Huiqi Li Linfeng Zhang 46 0 0 16 Mar 2025
CAKE: Cascading and Adaptive KV Cache Eviction with Layer Preferences Ziran Qin Yuchen Cao Mingbao Lin Wen Hu Shixuan Fan Ke Cheng Weiyao Lin Jianguo Li 61 3 0 16 Mar 2025
X-EcoMLA: Upcycling Pre-Trained Attention into MLA for Efficient and Extreme KV Compression Guihong Li Mehdi Rezagholizadeh Mingyu Yang Vikram Appia Emad Barsoum VLM 48 0 0 14 Mar 2025
ZSMerge: Zero-Shot KV Cache Compression for Memory-Efficient Long-Context LLMs Xin Liu Pei Liu Guoming Tang MoMe 44 0 0 13 Mar 2025
Training Plug-n-Play Knowledge Modules with Deep Context Distillation Lucas Page-Caccia Alan Ansell E. Ponti Ivan Vulić Alessandro Sordoni SyDa 82 0 0 11 Mar 2025
Multi-Cue Adaptive Visual Token Pruning for Large Vision-Language Models Bozhi Luan Wengang Zhou Hao Feng Zhe Wang Xiaosong Li H. Li VLM 61 0 0 11 Mar 2025
LLMs Know What to Drop: Self-Attention Guided KV Cache Eviction for Efficient Long-Context Inference G. Wang Shubhangi Upasani Chen Henry Wu Darshan Gandhi Jonathan Li Changran Hu Bo Li Urmish Thakker 75 0 0 11 Mar 2025
Dialogue Injection Attack: Jailbreaking LLMs through Context Manipulation Wenlong Meng Fan Zhang Wendao Yao Zhenyuan Guo Y. Li Chengkun Wei Wenzhi Chen AAML 36 1 0 11 Mar 2025
TokenButler: Token Importance is Predictable Yash Akhauri Ahmed F. AbouElhamayed Yifei Gao Chi-chih Chang Nilesh Jain Mohamed S. Abdelfattah 48 0 0 10 Mar 2025
Predicting Team Performance from Communications in Simulated Search-and-Rescue Ali Jalal-Kamali Nikolos Gurney David Pynadath AI4TS 102 8 0 05 Mar 2025
WeightedKV: Attention Scores Weighted Key-Value Cache Merging for Large Language Models Jian Yuan Ziwei He Haoli Bai Jingwen Leng Bo Jiang MoMe 34 0 0 03 Mar 2025
Dialogue Without Limits: Constant-Sized KV Caches for Extended Responses in LLMs Ravi Ghadia Avinash Kumar Gaurav Jain Prashant J. Nair Poulami Das 36 1 0 02 Mar 2025
Progressive Sparse Attention: Algorithm and System Co-design for Efficient Attention in LLM Serving Qihui Zhou Peiqi Yin Pengfei Zuo James Cheng CLL 35 1 0 01 Mar 2025
FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference Xunhao Lai Jianqiao Lu Yao Luo Yiyuan Ma Xun Zhou 63 5 0 28 Feb 2025
Long-Context Inference with Retrieval-Augmented Speculative Decoding Guanzheng Chen Qilong Feng Jinjie Ni Xin Li Michael Shieh RALM 47 2 0 27 Feb 2025
KVCrush: Key value cache size-reduction using similarity in head-behaviour Gopi Krishna Jha Sameh Gobriel Liubov Talamanova Alexander Kozlov Nilesh Jain MQ 29 0 0 24 Feb 2025
SVDq: 1.25-bit and 410x Key Cache Compression for LLM Attention Hong Yankun Li Xing Zhen Hui-Ling Yu Xianzhi Liu Wulong Yuan Mingxuan MQ 78 0 0 24 Feb 2025
The Lottery LLM Hypothesis, Rethinking What Abilities Should LLM Compression Preserve? Zhenheng Tang Xiang Liu Qian Wang Peijie Dong Bingsheng He Xiaowen Chu Bo Li LRM 50 1 0 24 Feb 2025
Neural Attention Search Difan Deng Marius Lindauer 85 0 0 21 Feb 2025
HeadInfer: Memory-Efficient LLM Inference by Head-wise Offloading Cheng Luo Zefan Cai Hanshi Sun Jinqi Xiao Bo Yuan Wen Xiao Junjie Hu Jiawei Zhao Beidi Chen Anima Anandkumar 56 1 0 18 Feb 2025
RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage Peter Yong Zhong Siyuan Chen Ruiqi Wang McKenna McCall Ben L. Titzer Heather Miller Phillip B. Gibbons LLMAG 85 3 0 17 Feb 2025