LLM-QAT: Data-Free Quantization Aware Training for Large Language Models

29 May 2023

Yashar Mehdad

Raghuraman Krishnamoorthi

Vikas Chandra

ArXiv PDF HTML

Papers citing "LLM-QAT: Data-Free Quantization Aware Training for Large Language Models"

50 / 151 papers shown

Title
LLM Inference Unveiled: Survey and Roofline Model Insights Zhihang Yuan Yuzhang Shang Yang Zhou Zhen Dong Zhe Zhou ... Yong Jae Lee Yan Yan Beidi Chen Guangyu Sun Kurt Keutzer 37 77 0 26 Feb 2024
MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases Zechun Liu Changsheng Zhao Forrest N. Iandola Chen Lai Yuandong Tian ... Ernie Chang Yangyang Shi Raghuraman Krishnamoorthi Liangzhen Lai Vikas Chandra ALM 30 68 0 22 Feb 2024
A Survey on Knowledge Distillation of Large Language Models Xiaohan Xu Ming Li Chongyang Tao Tao Shen Reynold Cheng Jinyang Li Can Xu Dacheng Tao Tianyi Zhou KELM VLM 34 94 0 20 Feb 2024
OneBit: Towards Extremely Low-bit Large Language Models Yuzhuang Xu Xu Han Zonghan Yang Shuo Wang Qingfu Zhu Zhiyuan Liu Weidong Liu Wanxiang Che MQ 45 36 0 17 Feb 2024
Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs Yeonhong Park Jake Hyun SangLyul Cho Bonggeun Sim Jae W. Lee MQ 25 16 0 16 Feb 2024
Model Compression and Efficient Inference for Large Language Models: A Survey Wenxiao Wang Wei Chen Yicong Luo Yongliu Long Zhengkai Lin Liye Zhang Binbin Lin Deng Cai Xiaofei He MQ 33 30 0 15 Feb 2024
RepQuant: Towards Accurate Post-Training Quantization of Large Transformer Models via Scale Reparameterization Zhikai Li Xuewen Liu Jing Zhang Qingyi Gu MQ 27 7 0 08 Feb 2024
Accurate LoRA-Finetuning Quantization of LLMs via Information Retention Haotong Qin Xudong Ma Xingyu Zheng Xiaoyang Li Yang Zhang Shouda Liu Jie Luo Xianglong Liu Michele Magno MQ 23 36 0 08 Feb 2024
L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Models Hyesung Jeon Yulhwa Kim Jae-Joon Kim MQ 19 4 0 07 Feb 2024
BiLLM: Pushing the Limit of Post-Training Quantization for LLMs Wei Huang Yangdong Liu Haotong Qin Ying Li Shiming Zhang Xianglong Liu Michele Magno Xiaojuan Qi MQ 77 63 0 06 Feb 2024
A Survey on Transformer Compression Yehui Tang Yunhe Wang Jianyuan Guo Zhijun Tu Kai Han Hailin Hu Dacheng Tao 21 26 0 05 Feb 2024
LQER: Low-Rank Quantization Error Reconstruction for LLMs Cheng Zhang Jianyi Cheng G. Constantinides Yiren Zhao MQ 14 8 0 04 Feb 2024
Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models Xindi Wang Mahsa Salmani Parsa Omidi Xiangyu Ren Mehdi Rezagholizadeh A. Eshaghi LRM 23 35 0 03 Feb 2024
Faster and Lighter LLMs: A Survey on Current Challenges and Way Forward Arnav Chavan Raghav Magazine Shubham Kushwaha M. Debbah Deepak Gupta 6 18 0 02 Feb 2024
KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization Coleman Hooper Sehoon Kim Hiva Mohammadzadeh Michael W. Mahoney Y. Shao Kurt Keutzer A. Gholami MQ 10 171 0 31 Jan 2024
Non-Vacuous Generalization Bounds for Large Language Models Sanae Lotfi Marc Finzi Yilun Kuang Tim G. J. Rudner Micah Goldblum Andrew Gordon Wilson 13 20 0 28 Dec 2023
Scaling Down to Scale Up: A Cost-Benefit Analysis of Replacing OpenAI's LLM with Open Source SLMs in Production Chandra Irugalbandara Ashish Mahendra Roland Daynauth T. Arachchige Jayanaka L. Dantanarayana K. Flautner Lingjia Tang Yiping Kang Jason Mars ELM 21 14 0 20 Dec 2023
CBQ: Cross-Block Quantization for Large Language Models Xin Ding Xiaoyu Liu Zhijun Tu Yun-feng Zhang Wei Li ... Hanting Chen Yehui Tang Zhiwei Xiong Baoqun Yin Yunhe Wang MQ 14 11 0 13 Dec 2023
SparQ Attention: Bandwidth-Efficient LLM Inference Luka Ribar Ivan Chelombiev Luke Hudlass-Galley Charlie Blake Carlo Luschi Douglas Orr 13 45 0 08 Dec 2023
SmoothQuant+: Accurate and Efficient 4-bit Post-Training WeightQuantization for LLM Jiayi Pan Chengcan Wang Kaifu Zheng Yangguang Li Zhenyu Wang Bin Feng MQ 25 7 0 06 Dec 2023
A Speed Odyssey for Deployable Quantization of LLMs Qingyuan Li Ran Meng Yiduo Li Bo-Wen Zhang Liang Li Yifan Lu Xiangxiang Chu Yerui Sun Yuchen Xie MQ 48 7 0 16 Nov 2023
SiDA-MoE: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Models Zhixu Du Shiyu Li Yuhao Wu Xiangyu Jiang Jingwei Sun Qilin Zheng Yongkai Wu Ang Li Hai Helen Li Yiran Chen MoE 10 11 0 29 Oct 2023
LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models Yixiao Li Yifan Yu Chen Liang Pengcheng He Nikos Karampatziakis Weizhu Chen Tuo Zhao MQ 28 117 0 12 Oct 2023
QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models Jing Liu Ruihao Gong Xiuying Wei Zhiwei Dong Jianfei Cai Bohan Zhuang MQ 12 30 0 12 Oct 2023
QFT: Quantized Full-parameter Tuning of LLMs with Affordable Resources Zhikai Li Xiaoxuan Liu Banghua Zhu Zhen Dong Qingyi Gu Kurt Keutzer MQ 19 7 0 11 Oct 2023
Compresso: Structured Pruning with Collaborative Prompting Learns Compact Large Language Models Song Guo Jiahang Xu Li Lyna Zhang Mao Yang 17 14 0 08 Oct 2023
Dual Grained Quantization: Efficient Fine-Grained Quantization for LLM Luoming Zhang Wen Fei Weijia Wu Yefei He Zhenyu Lou Hong Zhou MQ 9 5 0 07 Oct 2023
ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models Iman Mirzadeh Keivan Alizadeh-Vahid Sachin Mehta C. C. D. Mundo Oncel Tuzel Golnoosh Samei Mohammad Rastegari Mehrdad Farajtabar 118 58 0 06 Oct 2023
Compressing LLMs: The Truth is Rarely Pure and Never Simple Ajay Jaiswal Zhe Gan Xianzhi Du Bowen Zhang Zhangyang Wang Yinfei Yang MQ 26 45 0 02 Oct 2023
GrowLength: Accelerating LLMs Pretraining by Progressively Growing Training Length Hongye Jin Xiaotian Han Jingfeng Yang Zhimeng Jiang Chia-Yuan Chang Xia Hu 33 11 0 01 Oct 2023
PB-LLM: Partially Binarized Large Language Models Yuzhang Shang Zhihang Yuan Qiang Wu Zhen Dong MQ 4 43 0 29 Sep 2023
Rethinking Channel Dimensions to Isolate Outliers for Low-bit Weight Quantization of Large Language Models Jung Hwan Heo Jeonghoon Kim Beomseok Kwon Byeongwook Kim Se Jung Kwon Dongsoo Lee MQ 31 9 0 27 Sep 2023
QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models Yuhui Xu Lingxi Xie Xiaotao Gu Xin Chen Heng Chang Hengheng Zhang Zhensu Chen Xiaopeng Zhang Qi Tian MQ 11 87 0 26 Sep 2023
Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs Wenhua Cheng Weiwei Zhang Haihao Shen Yiyang Cai Xin He Kaokao Lv Yi. Liu MQ 8 21 0 11 Sep 2023
Understanding the Impact of Post-Training Quantization on Large Language Models Somnath Roy MQ 14 3 0 11 Sep 2023
Norm Tweaking: High-performance Low-bit Quantization of Large Language Models Liang Li Qingyuan Li Bo-Wen Zhang Xiangxiang Chu MQ 17 28 0 06 Sep 2023
eDKM: An Efficient and Accurate Train-time Weight Clustering for Large Language Models Minsik Cho Keivan Alizadeh Vahid Qichen Fu Saurabh N. Adya C. C. D. Mundo Mohammad Rastegari Devang Naik Peter Zatloukal MQ 13 6 0 02 Sep 2023
FPTQ: Fine-grained Post-Training Quantization for Large Language Models Qingyuan Li Yifan Zhang Liang Li Peng Yao Bo-Wen Zhang Xiangxiang Chu Yerui Sun Li-Qiang Du Yuchen Xie MQ 24 8 0 30 Aug 2023
Examining User-Friendly and Open-Sourced Large GPT Models: A Survey on Language, Multimodal, and Scientific GPT Models Kaiyuan Gao Su He Zhenyu He Jiacheng Lin Qizhi Pei Jie Shao Wei Zhang LM&MA SyDa 20 4 0 27 Aug 2023
OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models Wenqi Shao Mengzhao Chen Zhaoyang Zhang Peng-Tao Xu Lirui Zhao Zhiqiang Li Kaipeng Zhang Peng Gao Yu Qiao Ping Luo MQ 10 173 0 25 Aug 2023
A Survey on Model Compression for Large Language Models Xunyu Zhu Jian Li Yong Liu Can Ma Weiping Wang 11 98 0 15 Aug 2023
ZeroQuant-FP: A Leap Forward in LLMs Post-Training W4A8 Quantization Using Floating-Point Formats Xiaoxia Wu Z. Yao Yuxiong He MQ 16 43 0 19 Jul 2023
Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Study Peiyu Liu Zikang Liu Ze-Feng Gao Dawei Gao Wayne Xin Zhao Yaliang Li Bolin Ding Ji-Rong Wen MQ LRM 25 31 0 16 Jul 2023
A Comprehensive Overview of Large Language Models Humza Naveed Asad Ullah Khan Shi Qiu Muhammad Saqib Saeed Anwar Muhammad Usman Naveed Akhtar Nick Barnes Ajmal Saeed Mian OffRL 36 499 0 12 Jul 2023
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU Ying Sheng Lianmin Zheng Binhang Yuan Zhuohan Li Max Ryabinin ... Joseph E. Gonzalez Percy Liang Christopher Ré Ion Stoica Ce Zhang 138 208 0 13 Mar 2023
SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks Rui-Jie Zhu Qihang Zhao Guoqi Li Jason Eshraghian BDL VLM 14 80 0 27 Feb 2023
Quantization Robust Federated Learning for Efficient Inference on Heterogeneous Devices Kartik Gupta Marios Fournarakis M. Reisser Christos Louizos Markus Nagel FedML 6 14 0 22 Jun 2022
BiT: Robustly Binarized Multi-distilled Transformer Zechun Liu Barlas Oğuz Aasish Pappu Lin Xiao Scott Yih Meng Li Raghuraman Krishnamoorthi Yashar Mehdad MQ 26 50 0 25 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
BinaryBERT: Pushing the Limit of BERT Quantization Haoli Bai Wei Zhang Lu Hou Lifeng Shang Jing Jin Xin Jiang Qun Liu Michael Lyu Irwin King MQ 138 183 0 31 Dec 2020