ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition

23 February 2024

Papers citing "ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition"

7 / 7 papers shown

Title
Faster MoE LLM Inference for Extremely Large Models Haoqi Yang Luohe Shi Qiwei Li Zuchao Li Ping Wang Bo Du Mengjia Shen Hai Zhao MoE 59 0 0 06 May 2025
Cognitive Memory in Large Language Models Lianlei Shan Shixian Luo Zezhou Zhu Yu Yuan Yong Wu LLMAG KELM 69 1 0 03 Apr 2025
Twilight: Adaptive Attention Sparsity with Hierarchical Top- $p$ Pruning C. Lin Jiaming Tang Shuo Yang Hanshuo Wang Tian Tang Boyu Tian Ion Stoica Song Han Mingyu Gao 87 2 0 04 Feb 2025
FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving Zihao Ye Lequn Chen Ruihang Lai Wuwei Lin Yineng Zhang ... Tianqi Chen Baris Kasikci Vinod Grover Arvind Krishnamurthy Luis Ceze 65 19 0 02 Jan 2025
RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation Chao Jin Zili Zhang Xuanlin Jiang Fangyue Liu Xin Liu Xuanzhe Liu Xin Jin 32 36 0 18 Apr 2024
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU Ying Sheng Lianmin Zheng Binhang Yuan Zhuohan Li Max Ryabinin ... Joseph E. Gonzalez Percy Liang Christopher Ré Ion Stoica Ce Zhang 144 365 0 13 Mar 2023
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark A. Kalyan ELM ReLM LRM 207 1,089 0 20 Sep 2022