AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

1 June 2023

Chuang Gan

Song Han

EDL

ArXiv PDF HTML

Papers citing "AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration"

50 / 89 papers shown

Title
Semantic Retention and Extreme Compression in LLMs: Can We Have Both? Stanislas Laborde Martin Cousseau Antoun Yaacoub Lionel Prevost MQ 18 0 0 12 May 2025
LightNobel: Improving Sequence Length Limitation in Protein Structure Prediction Model via Adaptive Activation Quantization Seunghee Han S. Choi J. Kim 21 0 0 09 May 2025
Stability in Single-Peaked Strategic Resource Selection Games Henri Zeiler 21 3 0 09 May 2025
MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design Haojie Duanmu Xiuhong Li Zhihang Yuan Size Zheng Jiangfei Duan Xingcheng Zhang Dahua Lin MQ MoE 78 0 0 09 May 2025
Diffusion Model Quantization: A Review Qian Zeng Chenggong Hu Mingli Song Jie Song MQ 41 0 0 08 May 2025
DPQ-HD: Post-Training Compression for Ultra-Low Power Hyperdimensional Computing Nilesh Prasad Pandey Shriniwas Kulkarni David Wang Onat Gungor Flavio Ponzina T. Rosing 34 0 0 08 May 2025
Optimization Problem Solving Can Transition to Evolutionary Agentic Workflows Wenhao Li Bo Jin Mingyi Hong Changhong Lu Xiangfeng Wang 44 0 0 07 May 2025
Radio: Rate-Distortion Optimization for Large Language Model Compression Sean I. Young MQ 17 0 0 05 May 2025
Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniques Sanjay Surendranath Girija Shashank Kapoor Lakshit Arora Dipen Pradhan Aman Raj Ankit Shetgaonkar 47 0 0 05 May 2025
RetroInfer: A Vector-Storage Approach for Scalable Long-Context LLM Inference Y. Chen J. Zhang Baotong Lu Qianxi Zhang Chengruidong Zhang ... Chen Chen Mingxing Zhang Yuqing Yang Fan Yang Mao Yang 32 0 0 05 May 2025
Quantizing Diffusion Models from a Sampling-Aware Perspective Qian Zeng Jie Song Yuanyu Wan Huiqiong Wang Mingli Song DiffM MQ 71 1 0 04 May 2025
Real-time Spatial Retrieval Augmented Generation for Urban Environments David Nazareno Campo Javier Conde Alvaro Alonso Gabriel Huecas Joaquín Salvachúa Pedro Reviriego 17 0 0 04 May 2025
ICQuant: Index Coding enables Low-bit LLM Quantization Xinlin Li Osama A. Hanna Christina Fragouli Suhas Diggavi MQ 58 0 0 01 May 2025
Pushing the Limits of Low-Bit Optimizers: A Focus on EMA Dynamics Cong Xu Wenbin Liang Mo Yu Anan Liu K. Zhang Lizhuang Ma J. Wang J. Wang W. Zhang MQ 51 0 0 01 May 2025
FineQ: Software-Hardware Co-Design for Low-Bit Fine-Grained Mixed-Precision Quantization of LLMs Xilong Xie Liang Wang Limin Xiao Meng Han L. Sun S. Zheng Xiangrong Xu MQ 31 0 0 28 Apr 2025
TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate A. Zandieh Majid Daliri Majid Hadian Vahab Mirrokni MQ 74 0 0 28 Apr 2025
R-Sparse: Rank-Aware Activation Sparsity for Efficient LLM Inference Zhenyu (Allen) Zhang Zechun Liu Yuandong Tian Harshit Khaitan Z. Wang Steven Li 57 0 0 28 Apr 2025
Generative AI in Embodied Systems: System-Level Analysis of Performance, Efficiency and Scalability Zishen Wan Jiayi Qian Yuhang Du Jason J. Jabbour Yilun Du Yang Katie Zhao A. Raychowdhury Tushar Krishna Vijay Janapa Reddi LM&Ro 86 0 0 26 Apr 2025
Pushing the boundary on Natural Language Inference Pablo Miralles-González Javier Huertas-Tato Alejandro Martín David Camacho LRM 39 0 0 25 Apr 2025
PARD: Accelerating LLM Inference with Low-Cost PARallel Draft Model Adaptation Zihao An Huajun Bai Z. Liu Dong Li E. Barsoum 54 0 0 23 Apr 2025
$D$^{2}$MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving$ D $^{2}$ MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving Haodong Wang Qihua Zhou Zicong Hong Song Guo MoE 47 0 0 17 Apr 2025
CSPLADE: Learned Sparse Retrieval with Causal Language Models Zhichao Xu Aosong Feng Yijun Tian Haibo Ding Lin Leee Cheong RALM 40 0 0 15 Apr 2025
When Reasoning Meets Compression: Benchmarking Compressed Large Reasoning Models on Complex Reasoning Tasks Nan Zhang Yusen Zhang Prasenjit Mitra Rui Zhang MQ LRM 46 2 0 02 Apr 2025
Quamba2: A Robust and Scalable Post-training Quantization Framework for Selective State Space Models Hung-Yueh Chiang Chi-chih Chang N. Frumkin Kai-Chiang Wu Mohamed S. Abdelfattah Diana Marculescu MQ 69 0 0 28 Mar 2025
TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model Cheng Yang Yang Sui Jinqi Xiao Lingyi Huang Yu Gong ... Jinghua Yan Y. Bai P. Sadayappan Xia Hu Bo Yuan VLM 53 0 0 24 Mar 2025
Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models Yang Sui Yu-Neng Chuang Guanchu Wang Jiamu Zhang Tianyi Zhang ... Hongyi Liu Andrew Wen Shaochen Zhong Hanjie Chen OffRL ReLM LRM 74 21 0 20 Mar 2025
Triad: Empowering LMM-based Anomaly Detection with Vision Expert-guided Visual Tokenizer and Manufacturing Process Yuanze Li Shihao Yuan Haolin Wang Qizhang Li Ming-Yu Liu Chen Xu Guangming Shi Wangmeng Zuo 51 0 0 17 Mar 2025
PLM: Efficient Peripheral Language Models Hardware-Co-Designed for Ubiquitous Computing Cheng Deng Luoyang Sun Jiwen Jiang Yongcheng Zeng Xinjian Wu ... Haoyang Li Lei Chen Lionel M. Ni H. Zhang Jun Wang 83 0 0 15 Mar 2025
Are formal and functional linguistic mechanisms dissociated in language models? Michael Hanna Sandro Pezzelle Yonatan Belinkov 45 0 0 14 Mar 2025
Post-Training Quantization for Diffusion Transformer via Hierarchical Timestep Grouping Ning Ding Jing Han Yuchuan Tian Chao Xu Kai Han Yehui Tang MQ 42 0 0 10 Mar 2025
Wanda++: Pruning Large Language Models via Regional Gradients Yifan Yang Kai Zhen Bhavana Ganesh Aram Galstyan Goeric Huybrechts ... S. Bodapati Nathan Susanj Zheng Zhang Jack FitzGerald Abhishek Kumar 51 0 0 06 Mar 2025
PaCA: Partial Connection Adaptation for Efficient Fine-Tuning Sunghyeon Woo Sol Namkung Sunwoo Lee Inho Jeong Beomseok Kim Dongsuk Jeon 33 0 0 28 Feb 2025
A Sliding Layer Merging Method for Efficient Depth-Wise Pruning in LLMs Xuan Ding Rui Sun Yunjian Zhang Xiu Yan Yueqi Zhou Kaihao Huang Suzhong Fu Chuanlong Xie Chuanlong Xie Yao Zhu 69 1 0 26 Feb 2025
NEAT: Nonlinear Parameter-efficient Adaptation of Pre-trained Models Yibo Zhong Haoxiang Jiang Lincan Li Ryumei Nakada Tianci Liu Linjun Zhang Huaxiu Yao Haoyu Wang 75 2 0 24 Feb 2025
CORAL: Learning Consistent Representations across Multi-step Training with Lighter Speculative Drafter Yepeng Weng Dianwen Mei Huishi Qiu Xujie Chen Li Liu Jiang Tian Zhongchao Shi 44 0 0 24 Feb 2025
Dynamic Parallel Tree Search for Efficient LLM Reasoning Yifu Ding Wentao Jiang Shunyu Liu Yongcheng Jing J. Guo ... Zengmao Wang Z. Liu Bo Du X. Liu Dacheng Tao LRM 44 4 0 22 Feb 2025
Benchmarking Post-Training Quantization in LLMs: Comprehensive Taxonomy, Unified Evaluation, and Comparative Analysis J. Zhao M. Wang Miao Zhang Yuzhang Shang Xuebo Liu Yaowei Wang Min Zhang Liqiang Nie MQ 58 1 0 18 Feb 2025
Membership Inference Risks in Quantized Models: A Theoretical and Empirical Study Eric Aubinais Philippe Formont Pablo Piantanida Elisabeth Gassiat 38 0 0 10 Feb 2025
Division-of-Thoughts: Harnessing Hybrid Language Model Synergy for Efficient On-Device Agents Chenyang Shao Xinyuan Hu Yutang Lin Fengli Xu LLMAG LRM 59 4 0 06 Feb 2025
Position: AI Scaling: From Up to Down and Out Yunke Wang Yanxi Li Chang Xu HAI 74 1 0 02 Feb 2025
HyGen: Efficient LLM Serving via Elastic Online-Offline Request Co-location Ting Sun Penghan Wang Fan Lai 69 1 0 15 Jan 2025
Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoning Zhen Li Yupeng Su Runming Yang C. Xie Z. Wang Zhongwei Xie Ngai Wong Hongxia Yang MQ LRM 30 3 0 06 Jan 2025
LSAQ: Layer-Specific Adaptive Quantization for Large Language Model Deployment Binrui Zeng Bin Ji Xiaodong Liu Jie Yu Shasha Li Jun Ma Xiaopeng Li Shangwen Wang Xinran Hong Yongtao Tang MQ 36 1 0 24 Dec 2024
GQSA: Group Quantization and Sparsity for Accelerating Large Language Model Inference Chao Zeng Songwei Liu Shu Yang Fangmin Chen Xing Mei Lean Fu MQ 38 0 0 23 Dec 2024
Taming Sensitive Weights : Noise Perturbation Fine-tuning for Robust LLM Quantization Dongwei Wang Huanrui Yang MQ 82 1 0 08 Dec 2024
Unifying KV Cache Compression for Large Language Models with LeanKV Yanqi Zhang Yuwei Hu Runyuan Zhao John C. S. Lui Haibo Chen MQ 100 5 0 04 Dec 2024
BitMoD: Bit-serial Mixture-of-Datatype LLM Acceleration Yuzong Chen Ahmed F. AbouElhamayed Xilai Dai Yang Wang Marta Andronic G. Constantinides Mohamed S. Abdelfattah MQ 98 0 0 18 Nov 2024
EoRA: Training-free Compensation for Compressed LLM with Eigenspace Low-Rank Approximation Shih-yang Liu Huck Yang Nai Chit Fung Nai Chit Fung Hongxu Yin ... Jan Kautz Yu-Chun Wang Pavlo Molchanov Min-Hung Chen Min-Hung Chen MQ 29 0 0 28 Oct 2024
Self-calibration for Language Model Quantization and Pruning Miles Williams G. Chrysostomou Nikolaos Aletras MQ 54 0 0 22 Oct 2024
Opportunities and Challenges of Generative-AI in Finance Akshar Prabhu Desai Ganesh Satish Mallya Mohammad Luqman Tejasvi Ravi Nithya Kota Pranjul Yadav AIFin 31 2 0 21 Oct 2024