Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models

27 September 2022

Shanghang Zhang

Xianglong Liu

Papers citing "Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models"

50 / 112 papers shown

Title
Diffusion Model Quantization: A Review Qian Zeng Chenggong Hu Mingli Song Jie Song MQ 41 0 0 08 May 2025
EntroLLM: Entropy Encoded Weight Compression for Efficient Large Language Model Inference on Edge Devices Arnab Sanyal Prithwish Mukherjee Gourav Datta Sandeep P. Chinchali MQ 44 0 0 05 May 2025
Fast and Low-Cost Genomic Foundation Models via Outlier Removal Haozheng Luo Chenghao Qiu Maojiang Su Zhihan Zhou Zoe Mehta Guo Ye Jerry Yao-Chieh Hu Han Liu AAML 55 0 0 01 May 2025
GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration Yuhang Li Ruokai Yin Donghyun Lee Shiting Xiao Priyadarshini Panda MQ 45 0 0 03 Apr 2025
QUAD: Quantization and Parameter-Efficient Tuning of LLM with Activation Decomposition Yuxuan Hu Xiaodong Chen C. Li H. Chen J. Zhang MQ 60 0 0 25 Mar 2025
Post-Training Quantization for Diffusion Transformer via Hierarchical Timestep Grouping Ning Ding Jing Han Yuchuan Tian Chao Xu Kai Han Yehui Tang MQ 42 0 0 10 Mar 2025
Towards Superior Quantization Accuracy: A Layer-sensitive Approach Feng Zhang Yanbin Liu Weihua Li Jie Lv Xiaodan Wang Q. Bai MQ 44 0 0 09 Mar 2025
MergeQuant: Accurate 4-bit Static Quantization of Large Language Models by Channel-wise Calibration Jinguang Wang J. Wang Haifeng Sun Tingting Yang Zirui Zhuang Wanyi Ning Yuexi Yin Q. Qi Jianxin Liao MQ MoMe 44 0 0 07 Mar 2025
LightMamba: Efficient Mamba Acceleration on FPGA with Quantization and Hardware Co-design Renjie Wei Songqiang Xu Linfeng Zhong Zebin Yang Qingyu Guo Y. Wang Runsheng Wang Meng Li 74 0 0 24 Feb 2025
SpinQuant: LLM quantization with learned rotations Zechun Liu Changsheng Zhao Igor Fedorov Bilge Soran Dhruv Choudhary Raghuraman Krishnamoorthi Vikas Chandra Yuandong Tian Tijmen Blankevoort MQ 127 79 0 21 Feb 2025
PTQ1.61: Push the Real Limit of Extremely Low-Bit Post-Training Quantization Methods for Large Language Models J. Zhao Miao Zhang M. Wang Yuzhang Shang Kaihao Zhang Weili Guan Yaowei Wang Min Zhang MQ 39 0 0 18 Feb 2025
BCQ: Block Clustered Quantization for 4-bit (W4A4) LLM Inference Reena Elangovan Charbel Sakr A. Raghunathan Brucek Khailany MQ 42 1 0 07 Feb 2025
Improved Training Technique for Latent Consistency Models Quan Dao Khanh Doan Di Liu Trung Le Dimitris N. Metaxas 60 3 0 03 Feb 2025
Optimizing Large Language Model Training Using FP4 Quantization Ruizhe Wang Yeyun Gong Xiao Liu Guoshuai Zhao Ziyue Yang Baining Guo Zhengjun Zha Peng Cheng MQ 67 4 0 28 Jan 2025
PrefixQuant: Eliminating Outliers by Prefixed Tokens for Large Language Models Quantization Mengzhao Chen Yi Liu Jiahao Wang Yi Bin Wenqi Shao Ping Luo MQ 61 2 0 28 Jan 2025
Fast Matrix Multiplications for Lookup Table-Quantized LLMs Han Guo William Brandon Radostin Cholakov Jonathan Ragan-Kelley Eric P. Xing Yoon Kim MQ 73 12 0 20 Jan 2025
Taming Sensitive Weights : Noise Perturbation Fine-tuning for Robust LLM Quantization Dongwei Wang Huanrui Yang MQ 82 1 0 08 Dec 2024
CPTQuant -- A Novel Mixed Precision Post-Training Quantization Techniques for Large Language Models Amitash Nanda Sree Bhargavi Balija D. Sahoo MQ 59 0 0 03 Dec 2024
Anda: Unlocking Efficient LLM Inference with a Variable-Length Grouped Activation Data Format Chao Fang Man Shi Robin Geens Arne Symons Zhongfeng Wang Marian Verhelst 69 0 0 24 Nov 2024
Bi-Mamba: Towards Accurate 1-Bit State Space Models Shengkun Tang Liqun Ma H. Li Mingjie Sun Zhiqiang Shen Mamba 70 3 0 18 Nov 2024
The Super Weight in Large Language Models Mengxia Yu De Wang Qi Shan Colorado Reed Alvin Wan MQ MILM 29 9 0 11 Nov 2024
TesseraQ: Ultra Low-Bit LLM Post-Training Quantization with Block Reconstruction Yuhang Li Priyadarshini Panda MQ 26 1 0 24 Oct 2024
AERO: Softmax-Only LLMs for Efficient Private Inference N. Jha Brandon Reagen 25 1 0 16 Oct 2024
DAQ: Density-Aware Post-Training Weight-Only Quantization For LLMs Yingsong Luo Ling Chen MQ 16 0 0 16 Oct 2024
Channel-Wise Mixed-Precision Quantization for Large Language Models Zihan Chen Bike Xie Jundong Li Cong Shen MQ 22 1 0 16 Oct 2024
ReLU's Revival: On the Entropic Overload in Normalization-Free Large Language Models N. Jha Brandon Reagen OffRL AI4CE 28 0 0 12 Oct 2024
FlatQuant: Flatness Matters for LLM Quantization Yuxuan Sun Ruikang Liu Haoli Bai Han Bao Kang Zhao ... Lu Hou Chun Yuan Xin Jiang W. Liu Jun Yao MQ 50 3 0 12 Oct 2024
DARE the Extreme: Revisiting Delta-Parameter Pruning For Fine-Tuned Models Wenlong Deng Yize Zhao V. Vakilian Minghui Chen Xiaoxiao Li Christos Thrampoulidis 35 3 0 12 Oct 2024
CrossQuant: A Post-Training Quantization Method with Smaller Quantization Kernel for Precise Large Language Model Compression Wenyuan Liu Xindian Ma Peng Zhang Yan Wang MQ 24 0 0 10 Oct 2024
Q-VLM: Post-training Quantization for Large Vision-Language Models Changyuan Wang Ziwei Wang Xiuwei Xu Yansong Tang Jie Zhou Jiwen Lu MQ 27 1 0 10 Oct 2024
Scaling Laws for Mixed quantization in Large Language Models Zeyu Cao Cheng Zhang Pedro Gimenes Jianqiao Lu Jianyi Cheng Yiren Zhao MQ 29 1 0 09 Oct 2024
Gap Preserving Distillation by Building Bidirectional Mappings with A Dynamic Teacher Yong Guo Shulian Zhang Haolin Pan Jing Liu Yulun Zhang Jian Chen 30 0 0 05 Oct 2024
The Early Bird Catches the Leak: Unveiling Timing Side Channels in LLM Serving Systems Linke Song Zixuan Pang Wenhao Wang Zihao Wang XiaoFeng Wang Hongbo Chen Wei Song Yier Jin Dan Meng Rui Hou 40 6 0 30 Sep 2024
DopQ-ViT: Towards Distribution-Friendly and Outlier-Aware Post-Training Quantization for Vision Transformers Lianwei Yang Haisong Gong Qingyi Gu MQ 27 2 0 06 Aug 2024
Accurate and Efficient Fine-Tuning of Quantized Large Language Models Through Optimal Balance Ao Shen Qiang Wang Zhiquan Lai Xionglve Li Dongsheng Li ALM MQ 19 0 0 24 Jul 2024
Compensate Quantization Errors+: Quantized Models Are Inquisitive Learners Yifei Gao Jie Ou Lei Wang Fanhua Shang Jaji Wu Junguo Cheng MQ 22 0 0 22 Jul 2024
RazorAttention: Efficient KV Cache Compression Through Retrieval Heads Hanlin Tang Yang Lin Jing Lin Qingsen Han Shikuan Hong Yiwu Yao Gongyi Wang MQ 29 26 0 22 Jul 2024
RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization Xijie Huang Zechun Liu Shih-yang Liu Kwang-Ting Cheng MQ 35 7 0 10 Jul 2024
EfficientQAT: Efficient Quantization-Aware Training for Large Language Models Mengzhao Chen Wenqi Shao Peng Xu Jiahao Wang Peng Gao Kaipeng Zhang Yu Qiao Ping Luo MQ 36 21 0 10 Jul 2024
InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management Wonbeom Lee Jungi Lee Junghwan Seo Jaewoong Sim RALM 23 72 0 28 Jun 2024
OutlierTune: Efficient Channel-Wise Quantization for Large Language Models Jinguang Wang Yuexi Yin Haifeng Sun Qi Qi Jingyu Wang Zirui Zhuang Tingting Yang Jianxin Liao 33 2 0 27 Jun 2024
Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers Lei Chen Yuan Meng Chen Tang Xinzhu Ma Jingyan Jiang Xin Wang Zhi Wang Wenwu Zhu MQ 23 21 0 25 Jun 2024
Compensate Quantization Errors: Make Weights Hierarchical to Compensate Each Other Yifei Gao Jie Ou Lei Wang Yuting Xiao Zhiyuan Xiang Ruiting Dai Jun Cheng MQ 31 1 0 24 Jun 2024
Outlier Reduction with Gated Attention for Improved Post-training Quantization in Large Sequence-to-sequence Speech Foundation Models Dominik Wagner Ilja Baumann K. Riedhammer Tobias Bocklet MQ 25 1 0 16 Jun 2024
Tender: Accelerating Large Language Models via Tensor Decomposition and Runtime Requantization Jungi Lee Wonbeom Lee Jaewoong Sim MQ 21 14 0 16 Jun 2024
TernaryLLM: Ternarized Large Language Model Tianqi Chen Zhe Li Weixiang Xu Zeyu Zhu Dong Li Lu Tian E. Barsoum Peisong Wang Jian Cheng 28 7 0 11 Jun 2024
Low-Rank Quantization-Aware Training for LLMs Yelysei Bondarenko Riccardo Del Chiaro Markus Nagel MQ 33 8 0 10 Jun 2024
Evaluating Zero-Shot Long-Context LLM Compression Chenyu Wang Yihan Wang Kai Li 47 0 0 10 Jun 2024
Outliers and Calibration Sets have Diminishing Effect on Quantization of Modern LLMs Davide Paglieri Saurabh Dash Tim Rocktaschel Jack Parker-Holder MQ 34 6 0 31 May 2024
Understanding and Minimising Outlier Features in Neural Network Training Bobby He Lorenzo Noci Daniele Paliotta Imanol Schlag Thomas Hofmann 27 3 0 29 May 2024