SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models

18 November 2022

Song Han

Papers citing "SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models"

50 / 526 papers shown

Title
FP4DiT: Towards Effective Floating Point Quantization for Diffusion Transformers Ruichen Chen Keith G. Mills Di Niu MQ 52 0 0 19 Mar 2025
PIPO: Pipelined Offloading for Efficient Inference on Consumer Devices Yangyijian Liu Jun Yu Li Wu-Jun Li 31 0 0 15 Mar 2025
Towards Extreme Pruning of LLMs with Plug-and-Play Mixed Sparsity Chi Xu Gefei Zhang Yantong Zhu Luca Benini Guosheng Hu Yawei Li Zhihong Zhang 29 0 0 14 Mar 2025
Accurate INT8 Training Through Dynamic Block-Level Fallback Pengle Zhang Jia wei Jintao Zhang Jun-Jie Zhu Jianfei Chen MQ 74 3 0 13 Mar 2025
ZSMerge: Zero-Shot KV Cache Compression for Memory-Efficient Long-Context LLMs Xin Liu Pei Liu Guoming Tang MoMe 49 0 0 13 Mar 2025
OuroMamba: A Data-Free Quantization Framework for Vision Mamba Models Akshat Ramachandran Mingyu Lee Huan Xu Souvik Kundu Tushar Krishna MQ 49 1 0 13 Mar 2025
ViM-VQ: Efficient Post-Training Vector Quantization for Visual Mamba Juncan Deng Shuaiting Li Zeyu Wang Kedong Xu Hong Gu Kejie Huang MQ 60 0 0 12 Mar 2025
Mind the Memory Gap: Unveiling GPU Bottlenecks in Large-Batch LLM Inference Pol G. Recasens Ferran Agullo Yue Zhu Chen Wang Eun Kyung Lee Olivier Tardieu Jordi Torres Josep Ll. Berral 41 0 0 11 Mar 2025
Post-Training Quantization for Diffusion Transformer via Hierarchical Timestep Grouping Ning Ding Jing Han Yuchuan Tian Chao Xu Kai Han Yehui Tang MQ 42 0 0 10 Mar 2025
TR-DQ: Time-Rotation Diffusion Quantization Yihua Shao Deyang Lin Fanhu Zeng Minxi Yan M. Zhang ... Haozhe Wang J. Guo Yan Wang Haotong Qin Hao Tang MQ DiffM 72 1 0 09 Mar 2025
Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translation Yingfeng Luo Tong Zheng Yongyu Mu B. Li Qinghong Zhang ... Ziqiang Xu Peinan Feng Xiaoqian Liu Tong Xiao Jingbo Zhu AI4CE 103 0 0 09 Mar 2025
SAQ-SAM: Semantically-Aligned Quantization for Segment Anything Model Jing Zhang Z. Li Qingyi Gu MQ VLM 51 0 0 09 Mar 2025
MergeQuant: Accurate 4-bit Static Quantization of Large Language Models by Channel-wise Calibration Jinguang Wang J. Wang Haifeng Sun Tingting Yang Zirui Zhuang Wanyi Ning Yuexi Yin Q. Qi Jianxin Liao MQ MoMe 44 0 0 07 Mar 2025
Universality of Layer-Level Entropy-Weighted Quantization Beyond Model Architecture and Size Alireza Behtash Marijan Fofonjka Ethan Baird Tyler Mauer Hossein Moghimifam David Stout Joel Dennison MQ 53 1 0 06 Mar 2025
Knowledge-Decoupled Synergetic Learning: An MLLM based Collaborative Approach to Few-shot Multimodal Dialogue Intention Recognition Bin Chen Yu Zhang Hongfei Ye Ziyi Huang Hongyang Chen 62 1 0 06 Mar 2025
TeTRA-VPR: A Ternary Transformer Approach for Compact Visual Place Recognition Oliver Grainge Michael Milford I. Bodala Sarvapali D. Ramchurn Shoaib Ehsan ViT 62 0 0 04 Mar 2025
Identifying Sensitive Weights via Post-quantization Integral Yuezhou Hu Weiyu Huang Zichen Liang C. L. P. Chen Jintao Zhang J. Zhu Jianfei Chen MQ 39 2 0 28 Feb 2025
HALO: Hardware-aware quantization with low critical-path-delay weights for LLM acceleration Rohan Juneja Shivam Aggarwal Safeen Huda Tulika Mitra L. Peh 40 0 0 27 Feb 2025
Binary Neural Networks for Large Language Model: A Survey Liangdong Liu Zhitong Zheng Cong Wang Tianhuang Su Z. Yang MQ 65 0 0 26 Feb 2025
KVCrush: Key value cache size-reduction using similarity in head-behaviour Gopi Krishna Jha Sameh Gobriel Liubov Talamanova Alexander Kozlov Nilesh Jain MQ 34 0 0 24 Feb 2025
LightMamba: Efficient Mamba Acceleration on FPGA with Quantization and Hardware Co-design Renjie Wei Songqiang Xu Linfeng Zhong Zebin Yang Qingyu Guo Y. Wang Runsheng Wang Meng Li 79 0 0 24 Feb 2025
Dynamic Low-Rank Sparse Adaptation for Large Language Models Weizhong Huang Yuxin Zhang Xiawu Zheng Y. Liu Jing Lin Yiwu Yao Rongrong Ji 85 1 0 21 Feb 2025
SpinQuant: LLM quantization with learned rotations Zechun Liu Changsheng Zhao Igor Fedorov Bilge Soran Dhruv Choudhary Raghuraman Krishnamoorthi Vikas Chandra Yuandong Tian Tijmen Blankevoort MQ 127 79 0 21 Feb 2025
Hardware-Friendly Static Quantization Method for Video Diffusion Transformers Sanghyun Yi Qingfeng Liu Mostafa El-Khamy MQ VGen 35 0 0 20 Feb 2025
EvoP: Robust LLM Inference via Evolutionary Pruning Shangyu Wu Hongchao Du Ying Xiong Shuai Chen Tei-Wei Kuo Nan Guan Chun Jason Xue 34 1 0 19 Feb 2025
Efficient Long-Decoding Inference with Reasoning-Aware Attention Sparsity Junhao Hu Wenrui Huang Weidong Wang Zhenwen Li Tiancheng Hu Zhixia Liu Xusheng Chen Tao Xie Yizhou Shan LRM 43 0 0 16 Feb 2025
Skrr: Skip and Re-use Text Encoder Layers for Memory Efficient Text-to-Image Generation H. Seo Wongi Jeong Jae-sun Seo Se Young Chun 55 0 0 12 Feb 2025
Vertical Federated Learning in Practice: The Good, the Bad, and the Ugly Zhaomin Wu Zhen Qin Junyi Hou Haodong Zhao Qinbin Li Bingsheng He Lixin Fan FedML 64 1 0 12 Feb 2025
EfficientLLM: Scalable Pruning-Aware Pretraining for Architecture-Agnostic Edge Language Models Xingrun Xing Zheng Liu Shitao Xiao Boyan Gao Yiming Liang Wanpeng Zhang Haokun Lin Guoqi Li Jiajun Zhang LRM 56 1 0 10 Feb 2025
ParetoQ: Scaling Laws in Extremely Low-bit LLM Quantization Zechun Liu Changsheng Zhao Hanxian Huang Sijia Chen Jing Zhang ... Yuandong Tian Bilge Soran Raghuraman Krishnamoorthi Tijmen Blankevoort Vikas Chandra MQ 73 3 0 04 Feb 2025
Symmetric Pruning of Large Language Models Kai Yi Peter Richtárik AAML VLM 57 0 0 31 Jan 2025
Optimizing Large Language Model Training Using FP4 Quantization Ruizhe Wang Yeyun Gong Xiao Liu Guoshuai Zhao Ziyue Yang Baining Guo Zhengjun Zha Peng Cheng MQ 67 4 0 28 Jan 2025
PrefixQuant: Eliminating Outliers by Prefixed Tokens for Large Language Models Quantization Mengzhao Chen Yi Liu Jiahao Wang Yi Bin Wenqi Shao Ping Luo MQ 61 2 0 28 Jan 2025
GUIDE: A Global Unified Inference Engine for Deploying Large Language Models in Heterogeneous Environments Yanyu Chen Ganhong Huang 101 0 0 28 Jan 2025
HadamRNN: Binary and Sparse Ternary Orthogonal RNNs Armand Foucault Franck Mamalet François Malgouyres MQ 69 0 0 28 Jan 2025
OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitting Xing Hu Yuan Cheng Dawei Yang Zukang Xu Zhihang Yuan Jiangyong Yu Chen Xu Zhe Jiang Sifan Zhou MQ 36 4 0 23 Jan 2025
Irrational Complex Rotations Empower Low-bit Optimizers Zhen Tian Wayne Xin Zhao Ji-Rong Wen MQ 41 0 0 22 Jan 2025
DriveLM: Driving with Graph Visual Question Answering Chonghao Sima Katrin Renz Kashyap Chitta L. Chen Hanxue Zhang Chengen Xie Jens Beißwenger Ping Luo Andreas Geiger Hongyang Li 84 160 0 17 Jan 2025
Rethinking Post-Training Quantization: Introducing a Statistical Pre-Calibration Approach Alireza Ghaffari Sharareh Younesian Boxing Chen Vahid Partovi Nia M. Asgharian MQ 51 0 0 17 Jan 2025
HyGen: Efficient LLM Serving via Elastic Online-Offline Request Co-location Ting Sun Penghan Wang Fan Lai 98 1 0 15 Jan 2025
FlexQuant: Elastic Quantization Framework for Locally Hosted LLM on Edge Devices Yuji Chai Mujin Kwen David Brooks Gu-Yeon Wei MQ 29 3 0 13 Jan 2025
DiscQuant: A Quantization Method for Neural Networks Inspired by Discrepancy Theory Jerry Chee A. Backurs Rainie Heck Li Zhang Janardhan Kulkarni Thomas Rothvoss Sivakanth Gopi MQ 49 0 0 11 Jan 2025
Tensor Product Attention Is All You Need Yifan Zhang Yifeng Liu Huizhuo Yuan Zhen Qin Yang Yuan Q. Gu Andrew Chi-Chih Yao 75 9 0 11 Jan 2025
iServe: An Intent-based Serving System for LLMs Dimitrios Liakopoulos Tianrui Hu Prasoon Sinha N. Yadwadkar VLM 107 0 0 08 Jan 2025
Scaling Laws for Floating Point Quantization Training X. Sun Shuaipeng Li Ruobing Xie Weidong Han Kan Wu ... Yangyu Tao Zhanhui Kang C. Xu Di Wang Jie Jiang MQ AIFin 58 0 0 05 Jan 2025
FOLDER: Accelerating Multi-modal Large Language Models with Enhanced Performance Haicheng Wang Zhemeng Yu Gabriele Spadaro Chen Ju Victor Quétu Enzo Tartaglione Enzo Tartaglione VLM 88 3 0 05 Jan 2025
Pushing the Envelope of Low-Bit LLM via Dynamic Error Compensation Y. Park Jake Hyun Hojoon Kim Jae W. Lee MQ 36 0 0 31 Dec 2024
PTQ4VM: Post-Training Quantization for Visual Mamba Younghyun Cho Changhun Lee Seonggon Kim Eunhyeok Park MQ Mamba 34 2 0 29 Dec 2024
LSAQ: Layer-Specific Adaptive Quantization for Large Language Model Deployment Binrui Zeng Bin Ji Xiaodong Liu Jie Yu Shasha Li Jun Ma Xiaopeng Li Shangwen Wang Xinran Hong Yongtao Tang MQ 36 1 0 24 Dec 2024
TinyLLM: A Framework for Training and Deploying Language Models at the Edge Computers Savitha Viswanadh Kandala Pramuka Medaranga Ambuj Varshney 70 1 0 19 Dec 2024