What Makes Quantization for Large Language Models Hard? An Empirical
Study from the Lens of Perturbation

What Makes Quantization for Large Language Models Hard? An Empirical Study from the Lens of Perturbation

11 March 2024

Dongyan Zhao

Rui Yan

Papers citing "What Makes Quantization for Large Language Models Hard? An Empirical Study from the Lens of Perturbation"

4 / 4 papers shown

Title
Efficient Fine-Tuning of Quantized Models via Adaptive Rank and Bitwidth Changhai Zhou Yuhua Zhou Qian Qiao Weizhong Zhang Cheng Jin MQ 25 0 0 02 May 2025
ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation Z. Yao Xiaoxia Wu Cheng-rong Li Stephen Youn Yuxiong He MQ 63 56 0 15 Mar 2023
I-BERT: Integer-only BERT Quantization Sehoon Kim A. Gholami Z. Yao Michael W. Mahoney Kurt Keutzer MQ 86 332 0 05 Jan 2021
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT Sheng Shen Zhen Dong Jiayu Ye Linjian Ma Z. Yao A. Gholami Michael W. Mahoney Kurt Keutzer MQ 225 571 0 12 Sep 2019