Improving Neural Network Quantization without Retraining using Outlier Channel Splitting

28 January 2019

Papers citing "Improving Neural Network Quantization without Retraining using Outlier Channel Splitting"

50 / 174 papers shown

Title
Radio: Rate-Distortion Optimization for Large Language Model Compression Sean I. Young MQ 21 0 0 05 May 2025
GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration Yuhang Li Ruokai Yin Donghyun Lee Shiting Xiao Priyadarshini Panda MQ 48 0 0 03 Apr 2025
Improving Quantization with Post-Training Model Expansion Giuseppe Franco Pablo Monteagudo-Lago Ian Colbert Nicholas J. Fraser Michaela Blott MQ 57 1 0 21 Mar 2025
Task Vector Quantization for Memory-Efficient Model Merging Youngeun Kim Seunghwan Lee Aecheon Jung Bogon Ryu Sungeun Hong MQ MoMe 52 0 0 10 Mar 2025
SplitQuantV2: Enhancing Low-Bit Quantization of LLMs Without GPUs Jaewoo Song Fangzhen Lin MQ 51 0 0 07 Mar 2025
Improving Quantization-aware Training of Low-Precision Network via Block Replacement on Full-Precision Counterpart Chengting Yu Shu Yang Fengzhao Zhang Hanzhi Ma Aili Wang Er-ping Li MQ 77 2 0 20 Dec 2024
Post-Training Non-Uniform Quantization for Convolutional Neural Networks Ahmed Luqman Khuzemah Qazi Imdadullah Khan MQ 67 0 0 10 Dec 2024
On the Impact of White-box Deployment Strategies for Edge AI on Latency and Model Performance Jaskirat Singh Bram Adams Ahmed E. Hassan VLM 36 0 0 01 Nov 2024
Data Generation for Hardware-Friendly Post-Training Quantization Lior Dikstein Ariel Lapid Arnon Netzer H. Habi MQ 136 0 0 29 Oct 2024
Q-VLM: Post-training Quantization for Large Vision-Language Models Changyuan Wang Ziwei Wang Xiuwei Xu Yansong Tang Jie Zhou Jiwen Lu MQ 32 1 0 10 Oct 2024
QT-DoG: Quantization-aware Training for Domain Generalization Saqib Javed Hieu Le Mathieu Salzmann OOD MQ 28 1 0 08 Oct 2024
Foundations of Large Language Model Compression -- Part 1: Weight Quantization Sean I. Young MQ 40 1 0 03 Sep 2024
Infrared Domain Adaptation with Zero-Shot Quantization Burak Sevsay Erdem Akagündüz VLM MQ 30 1 0 25 Aug 2024
Low-Rank Quantization-Aware Training for LLMs Yelysei Bondarenko Riccardo Del Chiaro Markus Nagel MQ 33 10 0 10 Jun 2024
Nearest is Not Dearest: Towards Practical Defense against Quantization-conditioned Backdoor Attacks Boheng Li Yishuo Cai Haowei Li Feng Xue Zhifeng Li Yiming Li MQ AAML 27 20 0 21 May 2024
SwiftRL: Towards Efficient Reinforcement Learning on Real Processing-In-Memory Systems Kailash Gogineni Sai Santosh Dayapule Juan Gómez Luna Karthikeya Gogineni Peng Wei Tian-Shing Lan Mohammad Sadrosadati Onur Mutlu Guru Venkataramani 44 10 0 07 May 2024
Learning from Students: Applying t-Distributions to Explore Accurate and Efficient Formats for LLMs Jordan Dotzel Yuzong Chen Bahaa Kotb Sushma Prasad Gang Wu Sheng R. Li Mohamed S. Abdelfattah Zhiru Zhang 26 8 0 06 May 2024
Data-free Knowledge Distillation for Fine-grained Visual Categorization Renrong Shao Wei Zhang Jianhua Yin Jun Wang 31 2 0 18 Apr 2024
Parallel Decoding via Hidden Transfer for Lossless Large Language Model Acceleration Pengfei Wu Jiahao Liu Zhuocheng Gong Qifan Wang Jinpeng Li Jingang Wang Xunliang Cai Dongyan Zhao 20 1 0 18 Apr 2024
Minimize Quantization Output Error with Bias Compensation Cheng Gong Haoshuai Zheng Mengting Hu Zheng Lin Deng-Ping Fan Yuzhi Zhang Tao Li MQ 38 2 0 02 Apr 2024
On the Impact of Black-box Deployment Strategies for Edge AI on Latency and Model Performance Jaskirat Singh Emad Fallahzadeh Bram Adams Ahmed E. Hassan MQ 32 3 0 25 Mar 2024
ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models Chenyang Song Xu Han Zhengyan Zhang Shengding Hu Xiyu Shi ... Chen Chen Zhiyuan Liu Guanglin Li Tao Yang Maosong Sun 48 24 0 21 Feb 2024
Sequoia: Scalable, Robust, and Hardware-aware Speculative Decoding Zhuoming Chen Avner May Ruslan Svirschevski Yuhsun Huang Max Ryabinin Zhihao Jia Beidi Chen 34 37 0 19 Feb 2024
LR-CNN: Lightweight Row-centric Convolutional Neural Network Training for Memory Reduction Zhigang Wang Hangyu Yang Ning Wang Chuanfei Xu Jie Nie Zhiqiang Wei Yu Gu Ge Yu 13 0 0 21 Jan 2024
Hardware-Aware DNN Compression via Diverse Pruning and Mixed-Precision Quantization K. Balaskas Andreas Karatzas Christos Sad K. Siozios Iraklis Anagnostopoulos Georgios Zervakis Jörg Henkel MQ 33 10 0 23 Dec 2023
Mitigating Outlier Activations in Low-Precision Fine-Tuning of Language Models Alireza Ghaffari Justin Yu Mahsa Ghazvini Nejad M. Asgharian Boxing Chen Vahid Partovi Nia 13 2 0 14 Dec 2023
CBQ: Cross-Block Quantization for Large Language Models Xin Ding Xiaoyu Liu Zhijun Tu Yun-feng Zhang Wei Li ... Hanting Chen Yehui Tang Zhiwei Xiong Baoqun Yin Yunhe Wang MQ 27 13 0 13 Dec 2023
GenQ: Quantization in Low Data Regimes with Generative Synthetic Data Yuhang Li Youngeun Kim Donghyun Lee Souvik Kundu Priyadarshini Panda MQ 25 2 0 07 Dec 2023
PIPE : Parallelized Inference Through Post-Training Quantization Ensembling of Residual Expansions Edouard Yvinec Arnaud Dapogny Kévin Bailly MQ 10 0 0 27 Nov 2023
Shedding the Bits: Pushing the Boundaries of Quantization with Minifloats on FPGAs Shivam Aggarwal Hans Jakob Damsgaard Alessandro Pappalardo Giuseppe Franco Thomas B. Preußer Michaela Blott Tulika Mitra MQ 19 5 0 21 Nov 2023
Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time Zichang Liu Jue Wang Tri Dao Tianyi Zhou Binhang Yuan ... Anshumali Shrivastava Ce Zhang Yuandong Tian Christopher Ré Beidi Chen BDL 17 191 0 26 Oct 2023
Zero-Shot Sharpness-Aware Quantization for Pre-trained Language Models Miaoxi Zhu Qihuang Zhong Li Shen Liang Ding Juhua Liu Bo Du Dacheng Tao MQ VLM 29 1 0 20 Oct 2023
QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models Jing Liu Ruihao Gong Xiuying Wei Zhiwei Dong Jianfei Cai Bohan Zhuang MQ 23 51 0 12 Oct 2023
Network Memory Footprint Compression Through Jointly Learnable Codebooks and Mappings Vittorio Giammarino Arnaud Dapogny Kévin Bailly MQ 22 1 0 29 Sep 2023
Efficient Post-training Quantization with FP8 Formats Haihao Shen Naveen Mellempudi Xin He Q. Gao Chang‐Bao Wang Mengni Wang MQ 23 19 0 26 Sep 2023
SPFQ: A Stochastic Algorithm and Its Error Analysis for Neural Network Quantization Jinjie Zhang Rayan Saab 16 0 0 20 Sep 2023
On-Chip Hardware-Aware Quantization for Mixed Precision Neural Networks Wei Huang Haotong Qin Yangdong Liu Jingzhuo Liang Yifu Ding Ying Li Xianglong Liu MQ 23 0 0 05 Sep 2023
NUPES : Non-Uniform Post-Training Quantization via Power Exponent Search Edouard Yvinec Arnaud Dapogny Kévin Bailly MQ 24 6 0 10 Aug 2023
MRQ:Support Multiple Quantization Schemes through Model Re-Quantization Manasa Manohara Sankalp Dayal Tarqi Afzal Rahul Bakshi Kahkuen Fu MQ 20 0 0 01 Aug 2023
Self-Distilled Quantization: Achieving High Compression Rates in Transformer-Based Language Models James OÑeill Sourav Dutta VLM MQ 32 1 0 12 Jul 2023
Data-Free Quantization via Mixed-Precision Compensation without Fine-Tuning Jun Chen Shipeng Bai Tianxin Huang Mengmeng Wang Guanzhong Tian Y. Liu MQ 34 18 0 02 Jul 2023
H $_2$ O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models Zhenyu (Allen) Zhang Ying Sheng Tianyi Zhou Tianlong Chen Lianmin Zheng ... Yuandong Tian Christopher Ré Clark W. Barrett Zhangyang Wang Beidi Chen VLM 47 248 0 24 Jun 2023
Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing Yelysei Bondarenko Markus Nagel Tijmen Blankevoort MQ 13 88 0 22 Jun 2023
Training Transformers with 4-bit Integers Haocheng Xi Changhao Li Jianfei Chen Jun Zhu MQ 25 47 0 21 Jun 2023
ZeRO++: Extremely Efficient Collective Communication for Giant Model Training Guanhua Wang Heyang Qin S. A. Jacobs Connor Holmes Samyam Rajbhandari Olatunji Ruwase Feng Yan Lei Yang Yuxiong He VLM 55 57 0 16 Jun 2023
SqueezeLLM: Dense-and-Sparse Quantization Sehoon Kim Coleman Hooper A. Gholami Zhen Dong Xiuyu Li Sheng Shen Michael W. Mahoney Kurt Keutzer MQ 24 167 0 13 Jun 2023
FlexRound: Learnable Rounding based on Element-wise Division for Post-Training Quantization J. H. Lee Jeonghoon Kim S. Kwon Dongsoo Lee MQ 22 33 0 01 Jun 2023
PreQuant: A Task-agnostic Quantization Approach for Pre-trained Language Models Zhuocheng Gong Jiahao Liu Qifan Wang Yang Yang Jingang Wang Wei Yu Wu Yunsen Xian Dongyan Zhao Rui Yan MQ 33 5 0 30 May 2023
Towards Accurate Post-training Quantization for Diffusion Models Changyuan Wang Ziwei Wang Xiuwei Xu Yansong Tang Jie Zhou Jiwen Lu MQ 30 20 0 30 May 2023
Post-training Model Quantization Using GANs for Synthetic Data Generation Athanasios Masouris Mansi Sharma Adrian Boguszewski Alexander Kozlov Zhuo Wu Raymond Lo MQ 13 0 0 10 May 2023