Norm Tweaking: High-performance Low-bit Quantization of Large Language Models

6 September 2023

Papers citing "Norm Tweaking: High-performance Low-bit Quantization of Large Language Models"

23 / 23 papers shown

Title
Enhancing Ultra-Low-Bit Quantization of Large Language Models Through Saliency-Aware Partial Retraining Deyu Cao Samin Aref MQ 25 0 0 14 Apr 2025
RaanA: A Fast, Flexible, and Data-Efficient Post-Training Quantization Algorithm Yongyi Yang Jianyang Gao Wei Hu MQ 26 1 0 29 Mar 2025
Mobile-VideoGPT: Fast and Accurate Video Understanding Language Model Abdelrahman M. Shaker Muhammad Maaz Chenhui Gou Hamid Rezatofighi Salman Khan F. Khan 49 0 0 27 Mar 2025
Deploying Foundation Model Powered Agent Services: A Survey Wenchao Xu Jinyu Chen Peirong Zheng Xiaoquan Yi Tianyi Tian ... Quan Wan Haozhao Wang Yunfeng Fan Qinliang Su Xuemin Shen AI4CE 112 1 0 18 Dec 2024
Anda: Unlocking Efficient LLM Inference with a Variable-Length Grouped Activation Data Format Chao Fang Man Shi Robin Geens Arne Symons Zhongfeng Wang Marian Verhelst 69 0 0 24 Nov 2024
Demystifying Large Language Models for Medicine: A Primer Qiao Jin Nicholas Wan Robert Leaman Shubo Tian Zhizheng Wang ... Chunhua Weng Ronald M. Summers Qingyu Chen Yifan Peng Zhiyong Lu LM&MA 32 3 0 24 Oct 2024
Achieving Peak Performance for Large Language Models: A Systematic Review Z. R. K. Rostam Sándor Szénási Gábor Kertész 26 3 0 07 Sep 2024
EfficientQAT: Efficient Quantization-Aware Training for Large Language Models Mengzhao Chen Wenqi Shao Peng Xu Jiahao Wang Peng Gao Kaipeng Zhang Yu Qiao Ping Luo MQ 34 21 0 10 Jul 2024
I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models Xing Hu Yuan Cheng Dawei Yang Zhihang Yuan Jiangyong Yu Chen Xu Sifan Zhou MQ 25 6 0 28 May 2024
Integer Scale: A Free Lunch for Faster Fine-grained Quantization of LLMs Qingyuan Li Ran Meng Yiduo Li Bo Zhang Yifan Lu Yerui Sun Lin Ma Yuchen Xie MQ 33 0 0 23 May 2024
A Survey on Efficient Inference for Large Language Models Zixuan Zhou Xuefei Ning Ke Hong Tianyu Fu Jiaming Xu ... Shengen Yan Guohao Dai Xiao-Ping Zhang Yuhan Dong Yu-Xiang Wang 46 78 0 22 Apr 2024
VisionLLaMA: A Unified LLaMA Backbone for Vision Tasks Xiangxiang Chu Jianlin Su Bo-Wen Zhang Chunhua Shen MLLM 22 10 0 01 Mar 2024
LLM Inference Unveiled: Survey and Roofline Model Insights Zhihang Yuan Yuzhang Shang Yang Zhou Zhen Dong Zhe Zhou ... Yong Jae Lee Yan Yan Beidi Chen Guangyu Sun Kurt Keutzer 37 77 0 26 Feb 2024
HiRE: High Recall Approximate Top- $k$ Estimation for Efficient LLM Inference Yashas Samaga Varun Yerram Chong You Srinadh Bhojanapalli Sanjiv Kumar Prateek Jain Praneeth Netrapalli 33 4 0 14 Feb 2024
MobileVLM V2: Faster and Stronger Baseline for Vision Language Model Xiangxiang Chu Limeng Qiao Xinyu Zhang Shuang Xu Fei Wei ... Xiaofei Sun Yiming Hu Xinyang Lin Bo-Wen Zhang Chunhua Shen VLM MLLM 17 91 0 06 Feb 2024
KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization Coleman Hooper Sehoon Kim Hiva Mohammadzadeh Michael W. Mahoney Y. Shao Kurt Keutzer A. Gholami MQ 12 171 0 31 Jan 2024
MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devices Xiangxiang Chu Limeng Qiao Xinyang Lin Shuang Xu Yang Yang ... Fei Wei Xinyu Zhang Bo-Wen Zhang Xiaolin Wei Chunhua Shen MLLM 21 14 0 28 Dec 2023
PB-LLM: Partially Binarized Large Language Models Yuzhang Shang Zhihang Yuan Qiang Wu Zhen Dong MQ 11 43 0 29 Sep 2023
LORD: Low Rank Decomposition Of Monolingual Code LLMs For One-Shot Compression Ayush Kaushal Tejas Vaidhya Irina Rish 36 14 0 25 Sep 2023
Pruning Large Language Models via Accuracy Predictor Yupeng Ji Yibo Cao Jiu-si Liu KELM 19 4 0 18 Sep 2023
A Survey on Model Compression for Large Language Models Xunyu Zhu Jian Li Yong Liu Can Ma Weiping Wang 13 98 0 15 Aug 2023
ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation Z. Yao Xiaoxia Wu Cheng-rong Li Stephen Youn Yuxiong He MQ 63 56 0 15 Mar 2023
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU Ying Sheng Lianmin Zheng Binhang Yuan Zhuohan Li Max Ryabinin ... Joseph E. Gonzalez Percy Liang Christopher Ré Ion Stoica Ce Zhang 141 365 0 13 Mar 2023