Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers

25 May 2023

Papers citing "Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers"

45 / 45 papers shown

Title
Token Level Routing Inference System for Edge Devices Jianshu She Wenhao Zheng Zhengzhong Liu Hongyi Wang Eric P. Xing Huaxiu Yao Qirong Ho 36 0 0 10 Apr 2025
Adaptive Computation Pruning for the Forgetting Transformer Zhixuan Lin J. Obando-Ceron Xu Owen He Aaron C. Courville 30 0 0 09 Apr 2025
Saliency-driven Dynamic Token Pruning for Large Language Models Yao Tao Yehui Tang Yun Wang Mingjian Zhu Hailin Hu Yunhe Wang 32 0 0 06 Apr 2025
Cognitive Memory in Large Language Models Lianlei Shan Shixian Luo Zezhou Zhu Yu Yuan Yong Wu LLMAG KELM 63 1 0 03 Apr 2025
EfficientLLaVA:Generalizable Auto-Pruning for Large Vision-language Models Yinan Liang Z. Wang Xiuwei Xu Jie Zhou Jiwen Lu VLM LRM 46 0 0 19 Mar 2025
Key, Value, Compress: A Systematic Exploration of KV Cache Compression Techniques Neusha Javidnia B. Rouhani F. Koushanfar 49 0 0 14 Mar 2025
FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Compute Sotiris Anagnostidis Gregor Bachmann Yeongmin Kim Jonas Kohler Markos Georgopoulos A. Sanakoyeu Yuming Du Albert Pumarola Ali K. Thabet Edgar Schönfeld 76 0 0 27 Feb 2025
Neural Attention Search Difan Deng Marius Lindauer 85 0 0 21 Feb 2025
Position: AI Scaling: From Up to Down and Out Yunke Wang Yanxi Li Chang Xu HAI 71 1 0 02 Feb 2025
Deploying Foundation Model Powered Agent Services: A Survey Wenchao Xu Jinyu Chen Peirong Zheng Xiaoquan Yi Tianyi Tian ... Quan Wan Haozhao Wang Yunfeng Fan Qinliang Su Xuemin Shen AI4CE 112 1 0 18 Dec 2024
CSR:Achieving 1 Bit Key-Value Cache via Sparse Representation Hongxuan Zhang Yao Zhao Jiaqi Zheng Chenyi Zhuang Jinjie Gu Guihai Chen MQ 64 1 0 16 Dec 2024
MrT5: Dynamic Token Merging for Efficient Byte-level Language Models Julie Kallini Shikhar Murty Christopher D. Manning Christopher Potts Róbert Csordás 22 2 0 28 Oct 2024
Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Models Qitan Lv Jie Wang Hanzhu Chen Bin Li Yongdong Zhang Feng Wu HILM 17 3 0 19 Oct 2024
In-context KV-Cache Eviction for LLMs via Attention-Gate Zihao Zeng Bokai Lin Tianqi Hou Hao Zhang Zhijie Deng 23 1 0 15 Oct 2024
HSR-Enhanced Sparse Attention Acceleration Bo Chen Yingyu Liang Zhizhou Sha Zhenmei Shi Zhao-quan Song 75 17 0 14 Oct 2024
Selective Attention Improves Transformer Yaniv Leviathan Matan Kalman Yossi Matias 42 8 0 03 Oct 2024
A-VL: Adaptive Attention for Large Vision-Language Models Junyang Zhang Mu Yuan Ruiguang Zhong Puhan Luo Huiyou Zhan Ningkang Zhang Chengchen Hu Xiangyang Li VLM 36 1 0 23 Sep 2024
E2LLM: Encoder Elongated Large Language Models for Long-Context Understanding and Reasoning Zihan Liao Jun Wang Hang Yu Lingxiao Wei Jianguo Li Jun Wang Wei Zhang 19 2 0 10 Sep 2024
Post-Training Sparse Attention with Double Sparsity Shuo Yang Ying Sheng Joseph E. Gonzalez Ion Stoica Lianmin Zheng 18 7 0 11 Aug 2024
NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Time Yilong Chen Guoxia Wang Junyuan Shang Shiyao Cui Zhenyu Zhang Tingwen Liu Shuohuan Wang Yu Sun Dianhai Yu Hua-Hong Wu 18 14 0 07 Aug 2024
A2SF: Accumulative Attention Scoring with Forgetting Factor for Token Pruning in Transformer Decoder Hyun Rae Jo Dong Kun Shin 19 4 0 30 Jul 2024
LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference Qichen Fu Minsik Cho Thomas Merth Sachin Mehta Mohammad Rastegari Mahyar Najibi 28 25 0 19 Jul 2024
MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention Huiqiang Jiang Yucheng Li Chengruidong Zhang Qianhui Wu Xufang Luo ... Amir H. Abdi Dongsheng Li Chin-Yew Lin Yuqing Yang L. Qiu 65 1 0 02 Jul 2024
Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers Chao Lou Zixia Jia Zilong Zheng Kewei Tu ODL 26 18 0 24 Jun 2024
MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression Tianyu Fu Haofeng Huang Xuefei Ning Genghan Zhang Boju Chen ... Shiyao Li Shengen Yan Guohao Dai Huazhong Yang Yu Wang MQ 38 2 0 21 Jun 2024
CItruS: Chunked Instruction-aware State Eviction for Long Sequence Modeling Yu Bai Xiyuan Zou Heyan Huang Sanxing Chen Marc-Antoine Rondeau Yang Gao Jackie Chi Kit Cheung 27 3 0 17 Jun 2024
QCQA: Quality and Capacity-aware grouped Query Attention Vinay Joshi Prashant Laddha Shambhavi Sinha O. J. Omer S. Subramoney 16 4 0 08 Jun 2024
CORM: Cache Optimization with Recent Message for Large Language Model Inference Jincheng Dai Zhuowei Huang Haiyun Jiang Chen Chen Deng Cai Wei Bi Shuming Shi 19 3 0 24 Apr 2024
A Survey on Efficient Inference for Large Language Models Zixuan Zhou Xuefei Ning Ke Hong Tianyu Fu Jiaming Xu ... Shengen Yan Guohao Dai Xiao-Ping Zhang Yuhan Dong Yu-Xiang Wang 46 78 0 22 Apr 2024
SqueezeAttention: 2D Management of KV-Cache in LLM Inference via Layer-wise Optimal Budget Zihao Wang Shaoduo Gan 24 6 0 07 Apr 2024
Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inference Piotr Nawrot Adrian Lañcucki Marcin Chochowski David Tarjan E. Ponti 22 50 0 14 Mar 2024
Keyformer: KV Cache Reduction through Key Tokens Selection for Efficient Generative Inference Muhammad Adnan Akhil Arunkumar Gaurav Jain Prashant J. Nair Ilya Soloveychik Purushotham Kamath 16 52 0 14 Mar 2024
Fine-Grained Modeling of Narrative Context: A Coherence Perspective via Retrospective Questions Liyan Xu JiangNan Li Mo Yu Jie Zhou 17 3 0 21 Feb 2024
A Survey on Transformer Compression Yehui Tang Yunhe Wang Jianyuan Guo Zhijun Tu Kai Han Hailin Hu Dacheng Tao 24 26 0 05 Feb 2024
Transformers are Multi-State RNNs Matanel Oren Michael Hassid Nir Yarden Yossi Adi Roy Schwartz OffRL 19 34 0 11 Jan 2024
Attendre: Wait To Attend By Retrieval With Evicted Queries in Memory-Based Transformers for Long Context Processing Zi Yang Nan Hua RALM 29 4 0 10 Jan 2024
Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems Xupeng Miao Gabriele Oliaro Zhihao Zhang Xinhao Cheng Hongyi Jin Tianqi Chen Zhihao Jia 40 75 0 23 Dec 2023
Navigating Scaling Laws: Compute Optimality in Adaptive Model Training Sotiris Anagnostidis Gregor Bachmann Imanol Schlag Thomas Hofmann 17 2 0 06 Nov 2023
Efficient Streaming Language Models with Attention Sinks Michel Lang Yuandong Tian Beidi Chen Song Han Mike Lewis AI4TS RALM 11 629 0 29 Sep 2023
H $_2$ O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models Zhenyu (Allen) Zhang Ying Sheng Tianyi Zhou Tianlong Chen Lianmin Zheng ... Yuandong Tian Christopher Ré Clark W. Barrett Zhangyang Wang Beidi Chen VLM 32 246 0 24 Jun 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Latency Adjustable Transformer Encoder for Language Understanding Sajjad Kachuee M. Sharifkhani 21 0 0 10 Jan 2022
H-Transformer-1D: Fast One-Dimensional Hierarchical Attention for Sequences Zhenhai Zhu Radu Soricut 95 41 0 25 Jul 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 249 1,982 0 28 Jul 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 220 3,054 0 23 Jan 2020