GPTVQ: The Blessing of Dimensionality for LLM Quantization

23 February 2024

Papers citing "GPTVQ: The Blessing of Dimensionality for LLM Quantization"

26 / 26 papers shown

Title
GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance Jinuk Kim Marwa El Halabi W. Park Clemens JS Schaefer Deokjae Lee Yeonhong Park Jae W. Lee Hyun Oh Song MQ 19 0 0 11 May 2025
Radio: Rate-Distortion Optimization for Large Language Model Compression Sean I. Young MQ 17 0 0 05 May 2025
RWKVQuant: Quantizing the RWKV Family with Proxy Guided Hybrid of Scalar and Vector Quantization Chen Xu Yuxuan Yue Zukang Xu Xing Hu Jiangyong Yu Zhixuan Chen Sifan Zhou Zhihang Yuan Dawei Yang MQ 22 0 0 02 May 2025
MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidance Xing Hu Zhixuan Chen Dawei Yang Zukang Xu Chen Xu Zhihang Yuan Sifan Zhou Jiangyong Yu MoE MQ 27 0 0 02 May 2025
KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environments Junyoung Park Dalton Jones Matt Morse Raghavv Goel Mingu Lee Chris Lott 22 0 0 21 Apr 2025
NoWag: A Unified Framework for Shape Preserving Compression of Large Language Models Lawrence Liu Inesh Chakrabarti Yixiao Li Mengdi Wang Tuo Zhao Lin F. Yang MQ 21 0 0 20 Apr 2025
Achieving binary weight and activation for LLMs using Post-Training Quantization Siqing Song Chuang Wang Ruiqi Wang Yi Yang Xuyao Zhang MQ 26 0 0 07 Apr 2025
QUAD: Quantization and Parameter-Efficient Tuning of LLM with Activation Decomposition Yuxuan Hu Xiaodong Chen C. Li H. Chen J. Zhang MQ 60 0 0 25 Mar 2025
ClusComp: A Simple Paradigm for Model Compression and Efficient Finetuning Baohao Liao Christian Herold Seyyed Hadi Hashemi Stefan Vasilev Shahram Khadivi Christof Monz MQ 44 0 0 17 Mar 2025
ViM-VQ: Efficient Post-Training Vector Quantization for Visual Mamba Juncan Deng Shuaiting Li Zeyu Wang Kedong Xu Hong Gu Kejie Huang MQ 60 0 0 12 Mar 2025
SSVQ: Unleashing the Potential of Vector Quantization with Sign-Splitting Shuaiting Li Juncan Deng Chenxuan Wang Kedong Xu Rongtao Deng Hong Gu Haibin Shen Kejie Huang MQ 50 0 0 11 Mar 2025
SpinQuant: LLM quantization with learned rotations Zechun Liu Changsheng Zhao Igor Fedorov Bilge Soran Dhruv Choudhary Raghuraman Krishnamoorthi Vikas Chandra Yuandong Tian Tijmen Blankevoort MQ 127 76 0 21 Feb 2025
KVLink: Accelerating Large Language Models via Efficient KV Cache Reuse Jingbo Yang Bairu Hou Wei Wei Yujia Bao Shiyu Chang VLM 36 2 0 21 Feb 2025
BCQ: Block Clustered Quantization for 4-bit (W4A4) LLM Inference Reena Elangovan Charbel Sakr A. Raghunathan Brucek Khailany MQ 40 1 0 07 Feb 2025
ParetoQ: Scaling Laws in Extremely Low-bit LLM Quantization Zechun Liu Changsheng Zhao Hanxian Huang Sijia Chen Jing Zhang ... Yuandong Tian Bilge Soran Raghuraman Krishnamoorthi Tijmen Blankevoort Vikas Chandra MQ 73 2 0 04 Feb 2025
DiscQuant: A Quantization Method for Neural Networks Inspired by Discrepancy Theory Jerry Chee A. Backurs Rainie Heck Li Zhang Janardhan Kulkarni Thomas Rothvoss Sivakanth Gopi MQ 43 0 0 11 Jan 2025
Efficient LLM Inference using Dynamic Input Pruning and Cache-Aware Masking Marco Federici Davide Belli M. V. Baalen Amir Jalalirad Andrii Skliar Bence Major Markus Nagel Paul N. Whatmough 76 0 0 02 Dec 2024
Pushing the Limits of Large Language Model Quantization via the Linearity Theorem Vladimir Malinovskii Andrei Panferov Ivan Ilin Han Guo Peter Richtárik Dan Alistarh MQ 78 6 0 26 Nov 2024
TesseraQ: Ultra Low-Bit LLM Post-Training Quantization with Block Reconstruction Yuhang Li Priyadarshini Panda MQ 26 1 0 24 Oct 2024
Pyramid Vector Quantization for LLMs Tycho F. A. van der Ouderaa Maximilian L. Croci Agrin Hilmkil James Hensman MQ 24 0 0 22 Oct 2024
Residual vector quantization for KV cache compression in large language model Ankur Kumar MQ 29 0 0 21 Oct 2024
VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models Yifei Liu Jicheng Wen Yang Wang Shengyu Ye Li Lyna Zhang Ting Cao Cheng Li Mao Yang MQ 38 9 0 25 Sep 2024
GIFT-SW: Gaussian noise Injected Fine-Tuning of Salient Weights for LLMs Maxim Zhelnin Viktor Moskvoretskii Egor Shvetsov Egor Venediktov Mariya Krylova Aleksandr Zuev Evgeny Burnaev 19 2 0 27 Aug 2024
QTIP: Quantization with Trellises and Incoherence Processing Albert Tseng Qingyao Sun David Hou Christopher De Sa MQ 24 8 0 17 Jun 2024
PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression Vladimir Malinovskii Denis Mazur Ivan Ilin Denis Kuznedelev Konstantin Burlachenko Kai Yi Dan Alistarh Peter Richtárik MQ 24 18 0 23 May 2024
ApiQ: Finetuning of 2-Bit Quantized Large Language Model Baohao Liao Christian Herold Shahram Khadivi Christof Monz CLL MQ 34 12 0 07 Feb 2024