Low-Rank Prune-And-Factorize for Language Model Compression

25 June 2023

Papers citing "Low-Rank Prune-And-Factorize for Language Model Compression"

11 / 11 papers shown

Title
Forget the Data and Fine-Tuning! Just Fold the Network to Compress Dong Wang Haris Šikić Lothar Thiele O. Saukh 48 0 0 17 Feb 2025
LoRAP: Transformer Sub-Layers Deserve Differentiated Structured Compression for Large Language Models Guangyan Li Yongqiang Tang Wensheng Zhang 41 5 0 15 Apr 2024
Model Compression and Efficient Inference for Large Language Models: A Survey Wenxiao Wang Wei Chen Yicong Luo Yongliu Long Zhengkai Lin Liye Zhang Binbin Lin Deng Cai Xiaofei He MQ 38 47 0 15 Feb 2024
IoT in the Era of Generative AI: Vision and Challenges Xin Wang Zhongwei Wan Arvin Hekmati M. Zong Samiul Alam Mi Zhang Bhaskar Krishnamachari 27 15 0 03 Jan 2024
A Survey of Reasoning with Foundation Models Jiankai Sun Chuanyang Zheng E. Xie Zhengying Liu Ruihang Chu ... Xipeng Qiu Yi-Chen Guo Hui Xiong Qun Liu Zhenguo Li ReLM LRM AI4CE 24 76 0 17 Dec 2023
LORD: Low Rank Decomposition Of Monolingual Code LLMs For One-Shot Compression Ayush Kaushal Tejas Vaidhya Irina Rish 52 15 0 25 Sep 2023
$$\rm SP^3$: Enhancing Structured Pruning via PCA Projection$ $\rm SP^3$ : Enhancing Structured Pruning via PCA Projection Yuxuan Hu Jing Zhang Zhe Zhao Chengliang Zhao Xiaodong Chen Cuiping Li Hong Chen 28 1 0 31 Aug 2023
Distilling Linguistic Context for Language Model Compression Geondo Park Gyeongman Kim Eunho Yang 45 37 0 17 Sep 2021
The Lottery Ticket Hypothesis for Pre-trained BERT Networks Tianlong Chen Jonathan Frankle Shiyu Chang Sijia Liu Yang Zhang Zhangyang Wang Michael Carbin 150 345 0 23 Jul 2020
BERT-of-Theseus: Compressing BERT by Progressive Module Replacing Canwen Xu Wangchunshu Zhou Tao Ge Furu Wei Ming Zhou 221 197 0 07 Feb 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,956 0 20 Apr 2018