A2SF: Accumulative Attention Scoring with Forgetting Factor for Token Pruning in Transformer Decoder

30 July 2024

Papers citing "A2SF: Accumulative Attention Scoring with Forgetting Factor for Token Pruning in Transformer Decoder"

4 / 4 papers shown

Title
Cognitive Memory in Large Language Models Lianlei Shan Shixian Luo Zezhou Zhu Yu Yuan Yong Wu LLMAG KELM 75 1 0 03 Apr 2025
In-context KV-Cache Eviction for LLMs via Attention-Gate Zihao Zeng Bokai Lin Tianqi Hou Hao Zhang Zhijie Deng 38 1 0 15 Oct 2024
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU Ying Sheng Lianmin Zheng Binhang Yuan Zhuohan Li Max Ryabinin ... Joseph E. Gonzalez Percy Liang Christopher Ré Ion Stoica Ce Zhang 144 365 0 13 Mar 2023
Categorical Reparameterization with Gumbel-Softmax Eric Jang S. Gu Ben Poole BDL 75 5,262 0 03 Nov 2016