ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers

4 June 2022

Z. Yao

Reza Yazdani Aminabadi

Yuxiong He

Papers citing "ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers"

50 / 56 papers shown

Title
EntroLLM: Entropy Encoded Weight Compression for Efficient Large Language Model Inference on Edge Devices Arnab Sanyal Prithwish Mukherjee Gourav Datta Sandeep P. Chinchali MQ 37 0 0 05 May 2025
Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniques Sanjay Surendranath Girija Shashank Kapoor Lakshit Arora Dipen Pradhan Aman Raj Ankit Shetgaonkar 47 0 0 05 May 2025
Radio: Rate-Distortion Optimization for Large Language Model Compression Sean I. Young MQ 17 0 0 05 May 2025
Efficient Fine-Tuning of Quantized Models via Adaptive Rank and Bitwidth Changhai Zhou Yuhua Zhou Qian Qiao Weizhong Zhang Cheng Jin MQ 25 0 0 02 May 2025
FineQ: Software-Hardware Co-Design for Low-Bit Fine-Grained Mixed-Precision Quantization of LLMs Xilong Xie Liang Wang Limin Xiao Meng Han L. Sun S. Zheng Xiangrong Xu MQ 31 0 0 28 Apr 2025
Bi-directional Model Cascading with Proxy Confidence David Warren Mark Dras 41 0 0 27 Apr 2025
When Reasoning Meets Compression: Benchmarking Compressed Large Reasoning Models on Complex Reasoning Tasks Nan Zhang Yusen Zhang Prasenjit Mitra Rui Zhang MQ LRM 44 2 0 02 Apr 2025
Think Before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendation Jiakai Tang Sunhao Dai Teng Shi Jun Xu X. Chen Wen Chen Wu Jian Yuning Jiang LRM 56 5 0 28 Mar 2025
Striving for Simplicity: Simple Yet Effective Prior-Aware Pseudo-Labeling for Semi-Supervised Ultrasound Image Segmentation Yaxiong Chen Yujie Wang Zixuan Zheng Jingliang Hu Yilei Shi Shengwu Xiong Xiao Xiang Zhu Lichao Mou 49 1 0 18 Mar 2025
FedSpaLLM: Federated Pruning of Large Language Models Guangji Bai Yijiang Li Zilinghan Li Liang Zhao Kibaek Kim FedML 60 3 0 20 Feb 2025
Benchmarking Post-Training Quantization in LLMs: Comprehensive Taxonomy, Unified Evaluation, and Comparative Analysis J. Zhao M. Wang Miao Zhang Yuzhang Shang Xuebo Liu Yaowei Wang Min Zhang Liqiang Nie MQ 53 1 0 18 Feb 2025
HadamRNN: Binary and Sparse Ternary Orthogonal RNNs Armand Foucault Franck Mamalet François Malgouyres MQ 64 0 0 28 Jan 2025
LSAQ: Layer-Specific Adaptive Quantization for Large Language Model Deployment Binrui Zeng Bin Ji Xiaodong Liu Jie Yu Shasha Li Jun Ma Xiaopeng Li Shangwen Wang Xinran Hong Yongtao Tang MQ 36 1 0 24 Dec 2024
BitMoD: Bit-serial Mixture-of-Datatype LLM Acceleration Yuzong Chen Ahmed F. AbouElhamayed Xilai Dai Yang Wang Marta Andronic G. Constantinides Mohamed S. Abdelfattah MQ 93 0 0 18 Nov 2024
Scaling laws for post-training quantized large language models Zifei Xu Alexander Lan W. Yazar T. Webb Sayeh Sharify Xin Eric Wang MQ 21 0 0 15 Oct 2024
ARB-LLM: Alternating Refined Binarizations for Large Language Models Zhiteng Li X. Yan Tianao Zhang Haotong Qin Dong Xie Jiang Tian Zhongchao Shi Linghe Kong Yulun Zhang Xiaokang Yang MQ 26 2 0 04 Oct 2024
The Early Bird Catches the Leak: Unveiling Timing Side Channels in LLM Serving Systems Linke Song Zixuan Pang Wenhao Wang Zihao Wang XiaoFeng Wang Hongbo Chen Wei Song Yier Jin Dan Meng Rui Hou 38 6 0 30 Sep 2024
Foundations of Large Language Model Compression -- Part 1: Weight Quantization Sean I. Young MQ 32 1 0 03 Sep 2024
LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale Jaehong Cho Minsu Kim Hyunmin Choi Guseul Heo Jongse Park 32 8 0 10 Aug 2024
Accelerating Large Language Model Inference with Self-Supervised Early Exits Florian Valade LRM 36 1 0 30 Jul 2024
LRQ: Optimizing Post-Training Quantization for Large Language Models by Learning Low-Rank Weight-Scaling Matrices Jung Hyun Lee Jeonghoon Kim J. Yang S. Kwon Eunho Yang Kang Min Yoo Dongsoo Lee MQ 30 2 0 16 Jul 2024
Composable Interventions for Language Models Arinbjorn Kolbeinsson Kyle O'Brien Tianjin Huang Shanghua Gao Shiwei Liu ... Anurag J. Vaidya Faisal Mahmood Marinka Zitnik Tianlong Chen Thomas Hartvigsen KELM MU 73 5 0 09 Jul 2024
Let the Code LLM Edit Itself When You Edit the Code Zhenyu He Jun Zhang Shengjie Luo Jingjing Xu Z. Zhang Di He KELM 29 0 0 03 Jul 2024
LLMEasyQuant: Scalable Quantization for Parallel and Distributed LLM Inference Dong Liu Meng Jiang MQ 23 12 0 28 Jun 2024
BoA: Attention-aware Post-training Quantization without Backpropagation Junhan Kim Ho-Young Kim Eulrang Cho Chungman Lee Joonyoung Kim Yongkweon Jeon MQ 33 0 0 19 Jun 2024
Outlier Reduction with Gated Attention for Improved Post-training Quantization in Large Sequence-to-sequence Speech Foundation Models Dominik Wagner Ilja Baumann K. Riedhammer Tobias Bocklet MQ 22 1 0 16 Jun 2024
ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation Tianchen Zhao Tongcheng Fang Haofeng Huang Enshu Liu Widyadewi Soedarmadji ... Shengen Yan Huazhong Yang Xuefei Ning Xuefei Ning Yu Wang MQ VGen 94 21 0 04 Jun 2024
FastQuery: Communication-efficient Embedding Table Query for Private LLM Inference Chenqi Lin Tianshi Xu Zebin Yang Runsheng Wang Ru Huang Meng Li 14 0 0 25 May 2024
OAC: Output-adaptive Calibration for Accurate Post-training Quantization Ali Edalati Alireza Ghaffari M. Asgharian Lu Hou Boxing Chen Vahid Partovi Nia V. Nia MQ 78 0 0 23 May 2024
Characterizing the Accuracy - Efficiency Trade-off of Low-rank Decomposition in Language Models Chakshu Moar Michael Pellauer Hyoukjun Kwon 25 1 0 10 May 2024
Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment Abhinav Agarwalla Abhay Gupta Alexandre Marques Shubhra Pandit Michael Goin ... Tuan Nguyen Mahmoud Salem Dan Alistarh Sean Lie Mark Kurtz MoE SyDa 23 11 0 06 May 2024
Multilingual Brain Surgeon: Large Language Models Can be Compressed Leaving No Language Behind Hongchuan Zeng Hongshen Xu Lu Chen Kai Yu 44 4 0 06 Apr 2024
On the Challenges and Opportunities in Generative AI Laura Manduchi Kushagra Pandey Robert Bamler Ryan Cotterell Sina Daubener ... F. Wenzel Frank Wood Stephan Mandt Vincent Fortuin Vincent Fortuin 51 17 0 28 Feb 2024
Data-free Weight Compress and Denoise for Large Language Models Runyu Peng Yunhua Zhou Qipeng Guo Yang Gao Hang Yan Xipeng Qiu Dahua Lin 26 1 0 26 Feb 2024
Towards Meta-Pruning via Optimal Transport Alexander Theus Olin Geimer Friedrich Wicke Thomas Hofmann Sotiris Anagnostidis Sidak Pal Singh MoMe 6 3 0 12 Feb 2024
RepQuant: Towards Accurate Post-Training Quantization of Large Transformer Models via Scale Reparameterization Zhikai Li Xuewen Liu Jing Zhang Qingyi Gu MQ 27 7 0 08 Feb 2024
SmoothQuant+: Accurate and Efficient 4-bit Post-Training WeightQuantization for LLM Jiayi Pan Chengcan Wang Kaifu Zheng Yangguang Li Zhenyu Wang Bin Feng MQ 27 7 0 06 Dec 2023
Matrix Compression via Randomized Low Rank and Low Precision Factorization R. Saha Varun Srivastava Mert Pilanci 13 18 0 17 Oct 2023
Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs Yu-xin Zhang Lirui Zhao Mingbao Lin Yunyun Sun Yiwu Yao Xingjia Han Jared Tanner Shiwei Liu Rongrong Ji SyDa 24 40 0 13 Oct 2023
Rethinking Channel Dimensions to Isolate Outliers for Low-bit Weight Quantization of Large Language Models Jung Hwan Heo Jeonghoon Kim Beomseok Kwon Byeongwook Kim Se Jung Kwon Dongsoo Lee MQ 36 9 0 27 Sep 2023
Norm Tweaking: High-performance Low-bit Quantization of Large Language Models Liang Li Qingyuan Li Bo-Wen Zhang Xiangxiang Chu MQ 22 28 0 06 Sep 2023
Uncovering the Hidden Cost of Model Compression Diganta Misra Muawiz Chaudhary Agam Goyal Bharat Runwal Pin-Yu Chen VLM 24 0 0 29 Aug 2023
QuIP: 2-Bit Quantization of Large Language Models With Guarantees Jerry Chee Yaohui Cai Volodymyr Kuleshov Chris De Sa MQ 9 185 0 25 Jul 2023
A Simple and Effective Pruning Approach for Large Language Models Mingjie Sun Zhuang Liu Anna Bair J. Zico Kolter 35 350 0 20 Jun 2023
LLM-QAT: Data-Free Quantization Aware Training for Large Language Models Zechun Liu Barlas Oğuz Changsheng Zhao Ernie Chang Pierre Stock Yashar Mehdad Yangyang Shi Raghuraman Krishnamoorthi Vikas Chandra MQ 37 186 0 29 May 2023
Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers Sotiris Anagnostidis Dario Pavllo Luca Biggio Lorenzo Noci Aurélien Lucchi Thomas Hofmann 32 51 0 25 May 2023
A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation Xiaowei Huang Wenjie Ruan Wei Huang Gao Jin Yizhen Dong ... Sihao Wu Peipei Xu Dengyu Wu André Freitas Mustafa A. Mustafa ALM 27 81 0 19 May 2023
RPTQ: Reorder-based Post-training Quantization for Large Language Models Zhihang Yuan Lin Niu Jia-Wen Liu Wenyu Liu Xinggang Wang Yuzhang Shang Guangyu Sun Qiang Wu Jiaxiang Wu Bingzhe Wu MQ 14 75 0 03 Apr 2023
Fast DistilBERT on CPUs Haihao Shen Ofir Zafrir Bo Dong Hengyu Meng Xinyu. Ye Zhe Wang Yi Ding Hanwen Chang Guy Boudoukh Moshe Wasserblat VLM 11 2 0 27 Oct 2022
Towards Efficient Post-training Quantization of Pre-trained Language Models Haoli Bai Lu Hou Lifeng Shang Xin Jiang Irwin King M. Lyu MQ 44 47 0 30 Sep 2021