QuIP: 2-Bit Quantization of Large Language Models With Guarantees

25 July 2023

Papers citing "QuIP: 2-Bit Quantization of Large Language Models With Guarantees"

50 / 150 papers shown

Title
Understanding the Difficulty of Low-Precision Post-Training Quantization for LLMs Zifei Xu Sayeh Sharify W. Yazar T. Webb Xin Eric Wang MQ 28 0 0 18 Oct 2024
Channel-Wise Mixed-Precision Quantization for Large Language Models Zihan Chen Bike Xie Jundong Li Cong Shen MQ 22 2 0 16 Oct 2024
Skipping Computations in Multimodal LLMs Mustafa Shukor Matthieu Cord 24 2 0 12 Oct 2024
FlatQuant: Flatness Matters for LLM Quantization Yuxuan Sun Ruikang Liu Haoli Bai Han Bao Kang Zhao ... Lu Hou Chun Yuan Xin Jiang W. Liu Jun Yao MQ 57 3 0 12 Oct 2024
QuAILoRA: Quantization-Aware Initialization for LoRA Neal Lawton Aishwarya Padmakumar Judith Gaspers Jack FitzGerald Anoop Kumar Greg Ver Steeg Aram Galstyan MQ 24 0 0 09 Oct 2024
Scaling Laws for Mixed quantization in Large Language Models Zeyu Cao Cheng Zhang Pedro Gimenes Jianqiao Lu Jianyi Cheng Yiren Zhao MQ 29 1 0 09 Oct 2024
A Survey: Collaborative Hardware and Software Design in the Era of Large Language Models Cong Guo Feng Cheng Zhixu Du James Kiessling Jonathan Ku ... Qilin Zheng Guanglei Zhou Hai Li-Wei Li Yiran Chen 29 5 0 08 Oct 2024
QERA: an Analytical Framework for Quantization Error Reconstruction Cheng Zhang Jeffrey T. H. Wong Can Xiao G. Constantinides Yiren Zhao MQ 35 0 0 08 Oct 2024
QT-DoG: Quantization-aware Training for Domain Generalization Saqib Javed Hieu Le Mathieu Salzmann OOD MQ 26 1 0 08 Oct 2024
Accelerating Error Correction Code Transformers Matan Levy Yoni Choukroun Lior Wolf MQ 21 0 0 08 Oct 2024
Rotated Runtime Smooth: Training-Free Activation Smoother for accurate INT4 inference Ke Yi Zengke Liu Jianwei Zhang Chengyuan Li Tong Zhang Junyang Lin Jingren Zhou MQ 43 0 0 30 Sep 2024
Accumulator-Aware Post-Training Quantization Ian Colbert Fabian Grob Giuseppe Franco Jinjie Zhang Rayan Saab MQ 22 3 0 25 Sep 2024
VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models Yifei Liu Jicheng Wen Yang Wang Shengyu Ye Li Lyna Zhang Ting Cao Cheng Li Mao Yang MQ 41 9 0 25 Sep 2024
LLaMa-SciQ: An Educational Chatbot for Answering Science MCQ Marc-Antoine Allard Matin Ansaripour Maria Yuffa Paul Teiletche LRM 11 0 0 25 Sep 2024
MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Models Elias Frantar Roberto L. Castro Jiale Chen Torsten Hoefler Dan Alistarh MQ 14 10 0 21 Aug 2024
ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models Chao Zeng Songwei Liu Yusheng Xie Hong Liu Xiaojian Wang Miao Wei Shu Yang Fangmin Chen Xing Mei MQ 35 5 0 16 Aug 2024
STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMs Peijie Dong Lujun Li Dayou Du Yuhan Chen Zhenheng Tang ... Wei Xue Wenhan Luo Qi-fei Liu Yi-Ting Guo Xiaowen Chu MQ 40 4 0 03 Aug 2024
Unlocking Tokens as Data Points for Generalization Bounds on Larger Language Models Sanae Lotfi Yilun Kuang Brandon Amos Micah Goldblum Marc Finzi Andrew Gordon Wilson 19 7 0 25 Jul 2024
Compensate Quantization Errors+: Quantized Models Are Inquisitive Learners Yifei Gao Jie Ou Lei Wang Fanhua Shang Jaji Wu MQ 37 0 0 22 Jul 2024
PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation Branden Butler Sixing Yu Arya Mazaheri Ali Jannesari LRM 33 6 0 16 Jul 2024
LRQ: Optimizing Post-Training Quantization for Large Language Models by Learning Low-Rank Weight-Scaling Matrices Jung Hyun Lee Jeonghoon Kim J. Yang S. Kwon Eunho Yang Kang Min Yoo Dongsoo Lee MQ 36 2 0 16 Jul 2024
LeanQuant: Accurate Large Language Model Quantization with Loss-Error-Aware Grid Tianyi Zhang Anshumali Shrivastava MQ 23 2 0 14 Jul 2024
Optimization of DNN-based speaker verification model through efficient quantization technique Yeona Hong Woo-Jin Chung Hong-Goo Kang MQ 18 1 0 12 Jul 2024
FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision Jay Shah Ganesh Bikshandi Ying Zhang Vijay Thakkar Pradeep Ramani Tri Dao 48 112 0 11 Jul 2024
RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization Xijie Huang Zechun Liu Shih-yang Liu Kwang-Ting Cheng MQ 35 7 0 10 Jul 2024
EfficientQAT: Efficient Quantization-Aware Training for Large Language Models Mengzhao Chen Wenqi Shao Peng Xu Jiahao Wang Peng Gao Kaipeng Zhang Yu Qiao Ping Luo MQ 36 21 0 10 Jul 2024
VcLLM: Video Codecs are Secretly Tensor Codecs Ceyu Xu Yongji Wu Xinyu Yang Beidi Chen Matthew Lentz Danyang Zhuo Lisa Wu Wills 45 0 0 29 Jun 2024
ViT-1.58b: Mobile Vision Transformers in the 1-bit Era Zhengqing Yuan Rong-Er Zhou Hongyi Wang Lifang He Yanfang Ye Lichao Sun MQ 20 8 0 26 Jun 2024
Compensate Quantization Errors: Make Weights Hierarchical to Compensate Each Other Yifei Gao Jie Ou Lei Wang Yuting Xiao Zhiyuan Xiang Ruiting Dai Jun Cheng MQ 31 1 0 24 Jun 2024
Slice-Level Scheduling for High Throughput and Load Balanced LLM Serving Ke Cheng Wen Hu Zhi Wang Hongen Peng Jianguo Li Sheng Zhang 43 7 0 19 Jun 2024
BoA: Attention-aware Post-training Quantization without Backpropagation Junhan Kim Ho-Young Kim Eulrang Cho Chungman Lee Joonyoung Kim Yongkweon Jeon MQ 33 0 0 19 Jun 2024
Mixture of Scales: Memory-Efficient Token-Adaptive Binarization for Large Language Models Dongwon Jo Taesu Kim Yulhwa Kim Jae-Joon Kim 44 3 0 18 Jun 2024
QTIP: Quantization with Trellises and Incoherence Processing Albert Tseng Qingyao Sun David Hou Christopher De Sa MQ 32 8 0 17 Jun 2024
Evaluating the Generalization Ability of Quantized LLMs: Benchmark, Analysis, and Toolbox Yijun Liu Yuan Meng Fang Wu Shenhao Peng Hang Yao Chaoyu Guan Chen Tang Xinzhu Ma Zhi Wang Wenwu Zhu MQ 48 7 0 15 Jun 2024
QQQ: Quality Quattuor-Bit Quantization for Large Language Models Ying Zhang Peng Zhang Mincong Huang Jingyang Xiang Yujie Wang Chao Wang Yineng Zhang Lei Yu Chuan Liu Wei Lin VLM MQ 39 3 0 14 Jun 2024
TernaryLLM: Ternarized Large Language Model Tianqi Chen Zhe Li Weixiang Xu Zeyu Zhu Dong Li Lu Tian E. Barsoum Peisong Wang Jian Cheng 28 7 0 11 Jun 2024
Low-Rank Quantization-Aware Training for LLMs Yelysei Bondarenko Riccardo Del Chiaro Markus Nagel MQ 33 8 0 10 Jun 2024
PowerInfer-2: Fast Large Language Model Inference on a Smartphone Zhenliang Xue Yixin Song Zeyu Mi Le Chen Yubin Xia Haibo Chen 46 42 0 10 Jun 2024
ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization Haoran You Yipin Guo Yichao Fu Wei Zhou Huihong Shi Xiaofan Zhang Souvik Kundu Amir Yazdanbakhsh Y. Lin KELM 42 7 0 10 Jun 2024
Enabling Efficient Batch Serving for LMaaS via Generation Length Prediction Ke Cheng Wen Hu Zhi Wang Peng Du Jianguo Li Sheng Zhang 34 10 0 07 Jun 2024
Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity Wentao Guo Jikai Long Yimeng Zeng Zirui Liu Xinyu Yang ... Osbert Bastani Christopher De Sa Xiaodong Yu Beidi Chen Zhaozhuo Xu 26 14 0 05 Jun 2024
ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation Tianchen Zhao Tongcheng Fang Haofeng Huang Enshu Liu Widyadewi Soedarmadji ... Shengen Yan Huazhong Yang Xuefei Ning Xuefei Ning Yu Wang MQ VGen 97 22 0 04 Jun 2024
Achieving Sparse Activation in Small Language Models Jifeng Song Kai Huang Xiangyu Yin Boyuan Yang Wei Gao 29 4 0 03 Jun 2024
MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantization Aozhong Zhang Naigang Wang Yanxia Deng Xin Li Zi Yang Penghang Yin MQ 37 3 0 02 Jun 2024
Outliers and Calibration Sets have Diminishing Effect on Quantization of Modern LLMs Davide Paglieri Saurabh Dash Tim Rocktaschel Jack Parker-Holder MQ 34 6 0 31 May 2024
STAT: Shrinking Transformers After Training Megan Flynn Alexander Wang Dean Edward Alvarez Christopher De Sa Anil Damle 31 1 0 29 May 2024
Compressing Large Language Models using Low Rank and Low Precision Decomposition R. Saha Naomi Sagan Varun Srivastava Andrea J. Goldsmith Mert Pilanci MQ 16 7 0 29 May 2024
FinerCut: Finer-grained Interpretable Layer Pruning for Large Language Models Yang Zhang Yawei Li Xinpeng Wang Qianli Shen Barbara Plank Bernd Bischl Mina Rezaei Kenji Kawaguchi 47 7 0 28 May 2024
I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models Xing Hu Yuan Cheng Dawei Yang Zhihang Yuan Jiangyong Yu Chen Xu Sifan Zhou MQ 31 6 0 28 May 2024
CLAQ: Pushing the Limits of Low-Bit Post-Training Quantization for LLMs Haoyu Wang Bei Liu Hang Shao Bo Xiao Ke Zeng Guanglu Wan Yanmin Qian MQ 21 0 0 27 May 2024