Matrix Compression via Randomized Low Rank and Low Precision Factorization

17 October 2023

Papers citing "Matrix Compression via Randomized Low Rank and Low Precision Factorization"

13 / 13 papers shown

Title
Clustering-based Low-Rank Matrix Approximation: An Adaptive Theoretical Analysis with Application to Data Compression Sisipho Hamlomo M. Atemkeng 29 0 0 13 May 2025
Pushing the Envelope of Low-Bit LLM via Dynamic Error Compensation Y. Park Jake Hyun Hojoon Kim Jae W. Lee MQ 43 0 0 31 Dec 2024
Reassessing Layer Pruning in LLMs: New Insights and Methods Yao Lu Hao Cheng Yujie Fang Zeyu Wang Jiaheng Wei Dongwei Xu Qi Xuan Xiaoniu Yang Zhaowei Zhu 65 0 0 23 Nov 2024
FiRST: Finetuning Router-Selective Transformers for Input-Adaptive Latency Reduction Akriti Jain Saransh Sharma Koyel Mukherjee Soumyabrata Pal 31 1 0 16 Oct 2024
Getting Free Bits Back from Rotational Symmetries in LLMs Jiajun He Gergely Flamich José Miguel Hernández-Lobato MQ 21 0 0 02 Oct 2024
In-depth Analysis of Low-rank Matrix Factorisation in a Federated Setting Constantin Philippenko Kevin Scaman Laurent Massoulié FedML 29 1 0 13 Sep 2024
On-Device Language Models: A Comprehensive Review Jiajun Xu Zhiyuan Li Wei Chen Qun Wang Xin Gao Qi Cai Ziyuan Ling 44 27 0 26 Aug 2024
From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients Ajay Jaiswal Lu Yin Zhenyu (Allen) Zhang Shiwei Liu Jiawei Zhao Yuandong Tian Zhangyang Wang 33 14 0 15 Jul 2024
Compressing Large Language Models using Low Rank and Low Precision Decomposition R. Saha Naomi Sagan Varun Srivastava Andrea J. Goldsmith Mert Pilanci MQ 22 10 0 29 May 2024
A Survey on Efficient Inference for Large Language Models Zixuan Zhou Xuefei Ning Ke Hong Tianyu Fu Jiaming Xu ... Shengen Yan Guohao Dai Xiao-Ping Zhang Yuhan Dong Yu-Xiang Wang 46 83 0 22 Apr 2024
LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning Han Guo P. Greengard Eric P. Xing Yoon Kim MQ 36 43 0 20 Nov 2023
A Survey on Model Compression for Large Language Models Xunyu Zhu Jian Li Yong Liu Can Ma Weiping Wang 36 192 0 15 Aug 2023
DRIVE: One-bit Distributed Mean Estimation S. Vargaftik Ran Ben-Basat Amit Portnoy Gal Mendelson Y. Ben-Itzhak Michael Mitzenmacher OOD FedML 79 51 0 18 May 2021