SKVQ: Sliding-window Key and Value Cache Quantization for Large Language
Models

SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models

10 May 2024

Xingcheng Zhang

Dahua Lin

Papers citing "SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models"

3 / 3 papers shown

Title
MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design Haojie Duanmu Xiuhong Li Zhihang Yuan Size Zheng Jiangfei Duan Xingcheng Zhang Dahua Lin MQ MoE 75 0 0 09 May 2025
FlatQuant: Flatness Matters for LLM Quantization Yuxuan Sun Ruikang Liu Haoli Bai Han Bao Kang Zhao ... Lu Hou Chun Yuan Xin Jiang W. Liu Jun Yao MQ 57 3 0 12 Oct 2024
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU Ying Sheng Lianmin Zheng Binhang Yuan Zhuohan Li Max Ryabinin ... Joseph E. Gonzalez Percy Liang Christopher Ré Ion Stoica Ce Zhang 144 365 0 13 Mar 2023