Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models

27 September 2022

Shanghang Zhang

Xianglong Liu

Papers citing "Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models"

50 / 112 papers shown

Title
I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models Xing Hu Yuan Cheng Dawei Yang Zhihang Yuan Jiangyong Yu Chen Xu Sifan Zhou MQ 31 6 0 28 May 2024
CLAQ: Pushing the Limits of Low-Bit Post-Training Quantization for LLMs Haoyu Wang Bei Liu Hang Shao Bo Xiao Ke Zeng Guanglu Wan Yanmin Qian MQ 21 0 0 27 May 2024
PTQ4DiT: Post-training Quantization for Diffusion Transformers Junyi Wu Haoxuan Wang Yuzhang Shang Mubarak Shah Yan Yan MQ 33 18 0 25 May 2024
SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models Haojie Duanmu Zhihang Yuan Xiuhong Li Jiangfei Duan Xingcheng Zhang Dahua Lin MQ 27 18 0 10 May 2024
QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving Yujun Lin Haotian Tang Shang Yang Zhekai Zhang Guangxuan Xiao Chuang Gan Song Han 77 74 0 07 May 2024
PTQ4SAM: Post-Training Quantization for Segment Anything Chengtao Lv Hong Chen Jinyang Guo Yifu Ding Xianglong Liu VLM MQ 26 12 0 06 May 2024
Outlier-Efficient Hopfield Layers for Large Transformer-Based Models Jerry Yao-Chieh Hu Pei-Hsuan Chang Haozheng Luo Hong-Yu Chen Weijian Li Wei-Po Wang Han Liu 31 25 0 04 Apr 2024
Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization Aniruddha Nrusimha Mayank Mishra Naigang Wang Dan Alistarh Rameswar Panda Yoon Kim MQ 57 8 0 04 Apr 2024
Cherry on Top: Parameter Heterogeneity and Quantization in Large Language Models Wanyun Cui Qianle Wang MQ 31 1 0 03 Apr 2024
QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs Saleh Ashkboos Amirkeivan Mohtashami Maximilian L. Croci Bo Li Martin Jaggi Dan Alistarh Torsten Hoefler James Hensman MQ 32 123 0 30 Mar 2024
AffineQuant: Affine Transformation Quantization for Large Language Models Yuexiao Ma Huixia Li Xiawu Zheng Feng Ling Xuefeng Xiao Rui Wang Shilei Wen Fei Chao Rongrong Ji MQ 38 16 0 19 Mar 2024
QuantTune: Optimizing Model Quantization with Adaptive Outlier-Driven Fine Tuning Jiun-Man Chen Yu-Hsuan Chao Yu-Jie Wang Ming-Der Shieh Chih-Chung Hsu Wei-Fen Lin MQ 16 1 0 11 Mar 2024
EasyQuant: An Efficient Data-free Quantization Algorithm for LLMs Hanlin Tang Yifu Sun Decheng Wu Kai Liu Jianchen Zhu Zhanhui Kang MQ 21 9 0 05 Mar 2024
Evaluating Quantized Large Language Models Shiyao Li Xuefei Ning Luning Wang Tengxuan Liu Xiangsheng Shi Shengen Yan Guohao Dai Huazhong Yang Yu-Xiang Wang MQ 43 42 0 28 Feb 2024
EmMark: Robust Watermarks for IP Protection of Embedded Quantized Large Language Models Ruisi Zhang F. Koushanfar VLM WaLM 22 1 0 27 Feb 2024
Towards Accurate Post-training Quantization for Reparameterized Models Luoming Zhang Yefei He Wen Fei Zhenyu Lou Weijia Wu YangWei Ying Hong Zhou MQ 27 0 0 25 Feb 2024
WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More Yuxuan Yue Zhihang Yuan Haojie Duanmu Sifan Zhou Jianlong Wu Liqiang Nie MQ 22 41 0 19 Feb 2024
Model Compression and Efficient Inference for Large Language Models: A Survey Wenxiao Wang Wei Chen Yicong Luo Yongliu Long Zhengkai Lin Liye Zhang Binbin Lin Deng Cai Xiaofei He MQ 36 46 0 15 Feb 2024
RepQuant: Towards Accurate Post-Training Quantization of Large Transformer Models via Scale Reparameterization Zhikai Li Xuewen Liu Jing Zhang Qingyi Gu MQ 27 7 0 08 Feb 2024
QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuning Haoxuan Wang Yuzhang Shang Zhihang Yuan Junyi Wu Yan Yan DiffM MQ 11 28 0 06 Feb 2024
A Survey on Transformer Compression Yehui Tang Yunhe Wang Jianyuan Guo Zhijun Tu Kai Han Hailin Hu Dacheng Tao 24 26 0 05 Feb 2024
LQER: Low-Rank Quantization Error Reconstruction for LLMs Cheng Zhang Jianyi Cheng G. Constantinides Yiren Zhao MQ 19 8 0 04 Feb 2024
KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization Coleman Hooper Sehoon Kim Hiva Mohammadzadeh Michael W. Mahoney Y. Shao Kurt Keutzer A. Gholami MQ 12 171 0 31 Jan 2024
EPSD: Early Pruning with Self-Distillation for Efficient Model Compression Dong Chen Ning Liu Yichen Zhu Zhengping Che Rui Ma Fachao Zhang Xiaofeng Mou Yi Chang Jian Tang 13 3 0 31 Jan 2024
Building a Llama2-finetuned LLM for Odia Language Utilizing Domain Knowledge Instruction Set Guneet Singh Kohli Shantipriya Parida Sambit Sekhar Samirit Saha Nipun B. Nair Parul Agarwal Sonal Khosla Kusumlata Patiyal Debasish Dhal 19 13 0 19 Dec 2023
Mitigating Outlier Activations in Low-Precision Fine-Tuning of Language Models Alireza Ghaffari Justin Yu Mahsa Ghazvini Nejad M. Asgharian Boxing Chen Vahid Partovi Nia 13 2 0 14 Dec 2023
CBQ: Cross-Block Quantization for Large Language Models Xin Ding Xiaoyu Liu Zhijun Tu Yun-feng Zhang Wei Li ... Hanting Chen Yehui Tang Zhiwei Xiong Baoqun Yin Yunhe Wang MQ 19 11 0 13 Dec 2023
ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models Zhihang Yuan Yuzhang Shang Yue Song Qiang Wu Yan Yan Guangyu Sun MQ 29 41 0 10 Dec 2023
Efficient Quantization Strategies for Latent Diffusion Models Yuewei Yang Xiaoliang Dai Jialiang Wang Peizhao Zhang Hongbo Zhang DiffM MQ 17 13 0 09 Dec 2023
ZeroQuant-HERO: Hardware-Enhanced Robust Optimized Post-Training Quantization Framework for W8A8 Transformers Zhewei Yao Reza Yazdani Aminabadi Stephen Youn Xiaoxia Wu Elton Zheng Yuxiong He MQ 19 1 0 26 Oct 2023
QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models Jing Liu Ruihao Gong Xiuying Wei Zhiwei Dong Jianfei Cai Bohan Zhuang MQ 17 49 0 12 Oct 2023
PB-LLM: Partially Binarized Large Language Models Yuzhang Shang Zhihang Yuan Qiang Wu Zhen Dong MQ 11 43 0 29 Sep 2023
Rethinking Channel Dimensions to Isolate Outliers for Low-bit Weight Quantization of Large Language Models Jung Hwan Heo Jeonghoon Kim Beomseok Kwon Byeongwook Kim Se Jung Kwon Dongsoo Lee MQ 38 9 0 27 Sep 2023
QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models Yuhui Xu Lingxi Xie Xiaotao Gu Xin Chen Heng Chang Hengheng Zhang Zhensu Chen Xiaopeng Zhang Qi Tian MQ 13 87 0 26 Sep 2023
Efficient Post-training Quantization with FP8 Formats Haihao Shen Naveen Mellempudi Xin He Q. Gao Chang‐Bao Wang Mengni Wang MQ 20 19 0 26 Sep 2023
LORD: Low Rank Decomposition Of Monolingual Code LLMs For One-Shot Compression Ayush Kaushal Tejas Vaidhya Irina Rish 44 14 0 25 Sep 2023
Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs Wenhua Cheng Weiwei Zhang Haihao Shen Yiyang Cai Xin He Kaokao Lv Yi. Liu MQ 10 21 0 11 Sep 2023
QuantEase: Optimization-based Quantization for Language Models Kayhan Behdin Ayan Acharya Aman Gupta Qingquan Song Siyu Zhu S. Keerthi Rahul Mazumder MQ 14 19 0 05 Sep 2023
Memory Efficient Optimizers with 4-bit States Bingrui Li Jianfei Chen Jun Zhu MQ 17 32 0 04 Sep 2023
OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models Wenqi Shao Mengzhao Chen Zhaoyang Zhang Peng-Tao Xu Lirui Zhao Zhiqiang Li Kaipeng Zhang Peng Gao Yu Qiao Ping Luo MQ 10 173 0 25 Aug 2023
Token-Scaled Logit Distillation for Ternary Weight Generative Language Models Minsoo Kim Sihwa Lee Jangwhan Lee S. Hong Duhyeuk Chang Wonyong Sung Jungwook Choi MQ 16 14 0 13 Aug 2023
QuIP: 2-Bit Quantization of Large Language Models With Guarantees Jerry Chee Yaohui Cai Volodymyr Kuleshov Chris De Sa MQ 11 185 0 25 Jul 2023
Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing Yelysei Bondarenko Markus Nagel Tijmen Blankevoort MQ 6 84 0 22 Jun 2023
Training Transformers with 4-bit Integers Haocheng Xi Changhao Li Jianfei Chen Jun Zhu MQ 17 45 0 21 Jun 2023
A Simple and Effective Pruning Approach for Large Language Models Mingjie Sun Zhuang Liu Anna Bair J. Zico Kolter 35 350 0 20 Jun 2023
SqueezeLLM: Dense-and-Sparse Quantization Sehoon Kim Coleman Hooper A. Gholami Zhen Dong Xiuyu Li Sheng Shen Michael W. Mahoney Kurt Keutzer MQ 18 165 0 13 Jun 2023
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration Ji Lin Jiaming Tang Haotian Tang Shang Yang Wei-Ming Chen Wei-Chen Wang Guangxuan Xiao Xingyu Dang Chuang Gan Song Han EDL MQ 16 461 0 01 Jun 2023
Intriguing Properties of Quantization at Scale Arash Ahmadian Saurabh Dash Hongyu Chen Bharat Venkitesh Stephen Gou Phil Blunsom A. Ustun Sara Hooker MQ 29 38 0 30 May 2023
Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling Xiuying Wei Yunchen Zhang Yuhang Li Xiangguo Zhang Ruihao Gong Jian Ren Zhengang Li MQ 13 29 0 18 Apr 2023
SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks Rui-Jie Zhu Qihang Zhao Guoqi Li Jason Eshraghian BDL VLM 19 80 0 27 Feb 2023