ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models

16 August 2024

Papers citing "ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models"

7 / 7 papers shown

Title
Achieving binary weight and activation for LLMs using Post-Training Quantization Siqing Song Chuang Wang Ruiqi Wang Yi Yang Xuyao Zhang MQ 26 0 0 07 Apr 2025
Membership Inference Risks in Quantized Models: A Theoretical and Empirical Study Eric Aubinais Philippe Formont Pablo Piantanida Elisabeth Gassiat 38 0 0 10 Feb 2025
GQSA: Group Quantization and Sparsity for Accelerating Large Language Model Inference Chao Zeng Songwei Liu Shu Yang Fangmin Chen Xing Mei Lean Fu MQ 38 0 0 23 Dec 2024
SKIM: Any-bit Quantization Pushing The Limits of Post-Training Quantization Runsheng Bai Qiang Liu B. Liu MQ 59 1 0 05 Dec 2024
STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMs Peijie Dong Lujun Li Dayou Du Yuhan Chen Zhenheng Tang ... Wei Xue Wenhan Luo Qi-fei Liu Yi-Ting Guo Xiaowen Chu MQ 34 4 0 03 Aug 2024
QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks Albert Tseng Jerry Chee Qingyao Sun Volodymyr Kuleshov Christopher De Sa MQ 126 91 0 06 Feb 2024
BiLLM: Pushing the Limit of Post-Training Quantization for LLMs Wei Huang Yangdong Liu Haotong Qin Ying Li Shiming Zhang Xianglong Liu Michele Magno Xiaojuan Qi MQ 77 63 0 06 Feb 2024