v1v2v3v4v5 (latest)

SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantization

17 November 2024

ArXiv (abs)PDF HTML

Papers citing "SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantization"

8 / 8 papers shown

Title
SALE : Low-bit Estimation for Efficient Sparse Attention in Long-context LLM Prefilling Xiaodong Ji Hailin Zhang Fangcheng Fu Bin Cui 17 0 0 30 May 2025
QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache Rishabh Tiwari Haocheng Xi Aditya Tomar Coleman Hooper Sehoon Kim Maxwell Horton Mahyar Najibi Michael W. Mahoney Kemal Kurniawan Amir Gholami MQ 112 5 0 05 Feb 2025
Open-Sora: Democratizing Efficient Video Production for All Zangwei Zheng Xiangyu Peng Tianji Yang Chenhui Shen Shenggui Li Hongxin Liu Yukun Zhou Tianyi Li Yang You VGen 175 256 0 31 Dec 2024
COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training Haocheng Xi Han Cai Ligeng Zhu Yaojie Lu Kurt Keutzer Jianfei Chen Song Han MQ 168 11 0 25 Oct 2024
SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs Yizhao Gao Zhichen Zeng Dayou Du Shijie Cao Hayden Kwok-Hay So ... Junjie Lai Mao Yang Ting Cao Fan Yang M. Yang 148 28 0 17 Oct 2024
SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration Jintao Zhang Jia Wei Pengle Zhang Jun-Jie Zhu Jun Zhu Jianfei Chen VLM MQ 175 39 0 03 Oct 2024
CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer Zhuoyi Yang Jiayan Teng Wendi Zheng Ming Ding Shiyu Huang ... Weihan Wang Yean Cheng Xiaotao Gu Yuxiao Dong Jie Tang DiffM VGen 287 565 0 12 Aug 2024
QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving Chengyue Wu Haotian Tang Shang Yang Zhekai Zhang Guangxuan Xiao Chuang Gan Song Han 165 98 0 07 May 2024