LLM-QAT: Data-Free Quantization Aware Training for Large Language Models

29 May 2023

Yashar Mehdad

Raghuraman Krishnamoorthi

Vikas Chandra

ArXiv PDF HTML

Papers citing "LLM-QAT: Data-Free Quantization Aware Training for Large Language Models"

50 / 151 papers shown

Title
Quantitative Analysis of Performance Drop in DeepSeek Model Quantization Enbo Zhao Yi Shen Shuming Shi Jieyun Huang Z. Chen Ning Wang Siqi Xiao J. Zhang Kai Wang Shiguo Lian MQ 29 0 0 05 May 2025
Quantizing Diffusion Models from a Sampling-Aware Perspective Qian Zeng Jie Song Yuanyu Wan Huiqiong Wang Mingli Song DiffM MQ 61 1 0 04 May 2025
FineScope : Precision Pruning for Domain-Specialized Large Language Models Using SAE-Guided Self-Data Cultivation Chaitali Bhattacharyya Yeseong Kim 43 0 0 01 May 2025
PARD: Accelerating LLM Inference with Low-Cost PARallel Draft Model Adaptation Zihao An Huajun Bai Z. Liu Dong Li E. Barsoum 51 0 0 23 Apr 2025
Enhancing Ultra-Low-Bit Quantization of Large Language Models Through Saliency-Aware Partial Retraining Deyu Cao Samin Aref MQ 22 0 0 14 Apr 2025
Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization Yamato Arai Yuma Ichikawa MQ 24 0 0 13 Apr 2025
Task-Circuit Quantization: Leveraging Knowledge Localization and Interpretability for Compression Hanqi Xiao Yi-Lin Sung Elias Stengel-Eskin Mohit Bansal MQ 31 0 0 10 Apr 2025
Optimizing Large Language Models: Metrics, Energy Efficiency, and Case Study Insights Tahniat Khan Soroor Motie Sedef Akinli Kocak Shaina Raza MQ 27 0 0 07 Apr 2025
Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models Ruikang Liu Yuxuan Sun Manyi Zhang Haoli Bai Xianzhi Yu Tiezheng Yu C. Yuan Lu Hou MQ LRM 28 5 0 07 Apr 2025
When Reasoning Meets Compression: Benchmarking Compressed Large Reasoning Models on Complex Reasoning Tasks Nan Zhang Yusen Zhang Prasenjit Mitra Rui Zhang MQ LRM 44 2 0 02 Apr 2025
Model Hemorrhage and the Robustness Limits of Large Language Models Ziyang Ma Z. Li L. Zhang Gui-Song Xia Bo Du Liangpei Zhang Dacheng Tao 48 0 0 31 Mar 2025
Quamba2: A Robust and Scalable Post-training Quantization Framework for Selective State Space Models Hung-Yueh Chiang Chi-chih Chang N. Frumkin Kai-Chiang Wu Mohamed S. Abdelfattah Diana Marculescu MQ 46 0 0 28 Mar 2025
QUAD: Quantization and Parameter-Efficient Tuning of LLM with Activation Decomposition Yuxuan Hu Xiaodong Chen C. Li H. Chen J. Zhang MQ 58 0 0 25 Mar 2025
Accurate INT8 Training Through Dynamic Block-Level Fallback Pengle Zhang Jia wei Jintao Zhang Jun-Jie Zhu Jianfei Chen MQ 68 3 0 13 Mar 2025
Task Vector Quantization for Memory-Efficient Model Merging Youngeun Kim Seunghwan Lee Aecheon Jung Bogon Ryu Sungeun Hong MQ MoMe 49 0 0 10 Mar 2025
QArtSR: Quantization via Reverse-Module and Timestep-Retraining in One-Step Diffusion based Image Super-Resolution Libo Zhu Haotong Qin Kaicheng Yang W. J. Li Yong Guo Yulun Zhang Susanto Rahardja Xiaokang Yang MQ DiffM 59 0 0 07 Mar 2025
Identifying Sensitive Weights via Post-quantization Integral Yuezhou Hu Weiyu Huang Zichen Liang C. L. P. Chen Jintao Zhang J. Zhu Jianfei Chen MQ 37 2 0 28 Feb 2025
Binary Neural Networks for Large Language Model: A Survey Liangdong Liu Zhitong Zheng Cong Wang Tianhuang Su Z. Yang MQ 60 0 0 26 Feb 2025
KVCrush: Key value cache size-reduction using similarity in head-behaviour Gopi Krishna Jha Sameh Gobriel Liubov Talamanova Alexander Kozlov Nilesh Jain MQ 29 0 0 24 Feb 2025
SpinQuant: LLM quantization with learned rotations Zechun Liu Changsheng Zhao Igor Fedorov Bilge Soran Dhruv Choudhary Raghuraman Krishnamoorthi Vikas Chandra Yuandong Tian Tijmen Blankevoort MQ 124 76 0 21 Feb 2025
ParetoQ: Scaling Laws in Extremely Low-bit LLM Quantization Zechun Liu Changsheng Zhao Hanxian Huang Sijia Chen Jing Zhang ... Yuandong Tian Bilge Soran Raghuraman Krishnamoorthi Tijmen Blankevoort Vikas Chandra MQ 73 2 0 04 Feb 2025
Progressive Binarization with Semi-Structured Pruning for LLMs X. Yan Tianao Zhang Zhiteng Li Yulun Zhang MQ 44 0 0 03 Feb 2025
Optimizing Large Language Model Training Using FP4 Quantization Ruizhe Wang Yeyun Gong Xiao Liu Guoshuai Zhao Ziyue Yang Baining Guo Zhengjun Zha Peng Cheng MQ 64 4 0 28 Jan 2025
HadamRNN: Binary and Sparse Ternary Orthogonal RNNs Armand Foucault Franck Mamalet François Malgouyres MQ 64 0 0 28 Jan 2025
Irrational Complex Rotations Empower Low-bit Optimizers Zhen Tian Wayne Xin Zhao Ji-Rong Wen MQ 41 0 0 22 Jan 2025
Rethinking Post-Training Quantization: Introducing a Statistical Pre-Calibration Approach Alireza Ghaffari Sharareh Younesian Boxing Chen Vahid Partovi Nia M. Asgharian MQ 51 0 0 17 Jan 2025
Deploying Foundation Model Powered Agent Services: A Survey Wenchao Xu Jinyu Chen Peirong Zheng Xiaoquan Yi Tianyi Tian ... Quan Wan Haozhao Wang Yunfeng Fan Qinliang Su Xuemin Shen AI4CE 110 1 0 18 Dec 2024
Taming Sensitive Weights : Noise Perturbation Fine-tuning for Robust LLM Quantization Dongwei Wang Huanrui Yang MQ 74 1 0 08 Dec 2024
SKIM: Any-bit Quantization Pushing The Limits of Post-Training Quantization Runsheng Bai Qiang Liu B. Liu MQ 59 1 0 05 Dec 2024
PrefixKV: Adaptive Prefix KV Cache is What Vision Instruction-Following Models Need for Efficient Generation Ao Wang Hui Chen Jianchao Tan K. Zhang Xunliang Cai Zijia Lin J. Han Guiguang Ding VLM 77 3 0 04 Dec 2024
MiniKV: Pushing the Limits of LLM Inference via 2-Bit Layer-Discriminative KV Cache Akshat Sharma Hangliang Ding Jianping Li Neel Dani Minjia Zhang 69 1 0 27 Nov 2024
Anda: Unlocking Efficient LLM Inference with a Variable-Length Grouped Activation Data Format Chao Fang Man Shi Robin Geens Arne Symons Zhongfeng Wang Marian Verhelst 64 0 0 24 Nov 2024
Reassessing Layer Pruning in LLMs: New Insights and Methods Yao Lu Hao Cheng Yujie Fang Zeyu Wang Jiaheng Wei Dongwei Xu Qi Xuan Xiaoniu Yang Zhaowei Zhu 61 0 0 23 Nov 2024
FASTNav: Fine-tuned Adaptive Small-language-models Trained for Multi-point Robot Navigation Yuxuan Chen Yixin Han Xiao Li 57 1 0 20 Nov 2024
Llama Guard 3-1B-INT4: Compact and Efficient Safeguard for Human-AI Conversations Igor Fedorov Kate Plawiak Lemeng Wu Tarek Elgamal Naveen Suda ... Bilge Soran Zacharie Delpierre Coudert Rachad Alao Raghuraman Krishnamoorthi Vikas Chandra 59 2 0 18 Nov 2024
BitMoD: Bit-serial Mixture-of-Datatype LLM Acceleration Yuzong Chen Ahmed F. AbouElhamayed Xilai Dai Yang Wang Marta Andronic G. Constantinides Mohamed S. Abdelfattah MQ 93 0 0 18 Nov 2024
Artificial Intelligence for Biomedical Video Generation Linyuan Li Jianing Qiu Anujit Saha Lin Li Poyuan Li Mengxian He Ziyu Guo Wu Yuan VGen 55 1 0 12 Nov 2024
Scaling Laws for Precision Tanishq Kumar Zachary Ankner Benjamin Spector Blake Bordelon Niklas Muennighoff Mansheej Paul C. Pehlevan Christopher Ré Aditi Raghunathan AIFin MoMe 38 12 0 07 Nov 2024
A Comprehensive Study on Quantization Techniques for Large Language Models Jiedong Lang Zhehao Guo Shuyu Huang MQ 26 9 0 30 Oct 2024
EoRA: Training-free Compensation for Compressed LLM with Eigenspace Low-Rank Approximation Shih-yang Liu Huck Yang Nai Chit Fung Nai Chit Fung Hongxu Yin ... Jan Kautz Yu-Chun Wang Pavlo Molchanov Min-Hung Chen Min-Hung Chen MQ 29 0 0 28 Oct 2024
TesseraQ: Ultra Low-Bit LLM Post-Training Quantization with Block Reconstruction Yuhang Li Priyadarshini Panda MQ 26 1 0 24 Oct 2024
FiRST: Finetuning Router-Selective Transformers for Input-Adaptive Latency Reduction Akriti Jain Saransh Sharma Koyel Mukherjee Soumyabrata Pal 16 1 0 16 Oct 2024
Channel-Wise Mixed-Precision Quantization for Large Language Models Zihan Chen Bike Xie Jundong Li Cong Shen MQ 17 1 0 16 Oct 2024
Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMs Ruijia Niu D. Wu Rose Yu Yi-An Ma 18 1 0 09 Oct 2024
QERA: an Analytical Framework for Quantization Error Reconstruction Cheng Zhang Jeffrey T. H. Wong Can Xiao G. Constantinides Yiren Zhao MQ 30 0 0 08 Oct 2024
ParallelSpec: Parallel Drafter for Efficient Speculative Decoding Zilin Xiao Hongming Zhang Tao Ge Siru Ouyang Vicente Ordonez Dong Yu 39 5 0 08 Oct 2024
Mixture Compressor for Mixture-of-Experts LLMs Gains More Wei Huang Yue Liao Jianhui Liu Ruifei He Haoru Tan Shiming Zhang Hongsheng Li Si Liu Xiaojuan Qi MoE 36 3 0 08 Oct 2024
Continuous Approximations for Improving Quantization Aware Training of LLMs He Li Jianhang Hong Yuanzhuo Wu Snehal Adbol Zonglin Li MQ 11 0 0 06 Oct 2024
Gap Preserving Distillation by Building Bidirectional Mappings with A Dynamic Teacher Yong Guo Shulian Zhang Haolin Pan Jing Liu Yulun Zhang Jian Chen 25 0 0 05 Oct 2024
ARB-LLM: Alternating Refined Binarizations for Large Language Models Zhiteng Li X. Yan Tianao Zhang Haotong Qin Dong Xie Jiang Tian Zhongchao Shi Linghe Kong Yulun Zhang Xiaokang Yang MQ 13 2 0 04 Oct 2024