Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes

Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes

8 February 2024

Steven Kolawole

Jean-Francois Kagey

Graham Neubig

Ameet Talwalkar

Papers citing "Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes"

18 / 18 papers shown

Title
Sparsity Outperforms Low-Rank Projections in Few-Shot Adaptation Nairouz Mrabah Nicolas Richet Ismail Ben Ayed Eric Granger BDL VLM 47 0 0 16 Apr 2025
Model Hemorrhage and the Robustness Limits of Large Language Models Ziyang Ma Z. Li L. Zhang Gui-Song Xia Bo Du Liangpei Zhang Dacheng Tao 50 0 0 31 Mar 2025
SEAP: Training-free Sparse Expert Activation Pruning Unlock the Brainpower of Large Language Models Xun Liang Hanyu Wang Huayi Lai Simin Niu Shichao Song Jiawei Yang Jihao Zhao Feiyu Xiong Bo Tang Z. Li VLM 40 0 0 10 Mar 2025
MaskPrune: Mask-based LLM Pruning for Layer-wise Uniform Structures Jiayu Qin Jianchao Tan K. Zhang Xunliang Cai Wei Wang 40 0 0 19 Feb 2025
Pruning-aware Loss Functions for STOI-Optimized Pruned Recurrent Autoencoders for the Compression of the Stimulation Patterns of Cochlear Implants at Zero Delay Reemt Hinrichs Jörn Ostermann 86 0 0 04 Feb 2025
Zeroth-Order Adaptive Neuron Alignment Based Pruning without Re-Training Elia Cunegatti Leonardo Lucio Custode Giovanni Iacca 36 0 0 11 Nov 2024
ESPACE: Dimensionality Reduction of Activations for Model Compression Charbel Sakr Brucek Khailany 15 2 0 07 Oct 2024
CFSP: An Efficient Structured Pruning Framework for LLMs with Coarse-to-Fine Activation Information Yuxin Wang Minghua Ma Zekun Wang Jingchang Chen Huiming Fan Liping Shan Qing Yang Dongliang Xu Ming Liu Bing Qin 16 3 0 20 Sep 2024
STUN: Structured-Then-Unstructured Pruning for Scalable MoE Pruning Jaeseong Lee seung-won hwang Aurick Qiao Daniel F Campos Z. Yao Yuxiong He 18 2 0 10 Sep 2024
Compact Language Models via Pruning and Knowledge Distillation Saurav Muralidharan Sharath Turuvekere Sreenivas Raviraj Joshi Marcin Chochowski M. Patwary M. Shoeybi Bryan Catanzaro Jan Kautz Pavlo Molchanov SyDa MQ 27 36 0 19 Jul 2024
VTrans: Accelerating Transformer Compression with Variational Information Bottleneck based Pruning Oshin Dutta Ritvik Gupta Sumeet Agarwal 31 1 0 07 Jun 2024
On Fairness of Low-Rank Adaptation of Large Models Zhoujie Ding Ken Ziyu Liu Pura Peetathawatchai Berivan Isik Sanmi Koyejo 38 4 0 27 May 2024
Your Transformer is Secretly Linear Anton Razzhigaev Matvey Mikhalchuk Elizaveta Goncharova Nikolai Gerasimenko Ivan V. Oseledets Denis Dimitrov Andrey Kuznetsov 22 4 0 19 May 2024
Rethinking Pruning for Vision-Language Models: Strategies for Effective Sparsity and Performance Restoration Shwai He Ang Li Tianlong Chen VLM 26 1 0 03 Apr 2024
Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing? Marco Gaido Sara Papi Matteo Negri L. Bentivogli 38 11 0 19 Feb 2024
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes Lokesh Nagalapatti Chun-Liang Li Chih-Kuan Yeh Hootan Nakhost Yasuhisa Fujii Alexander Ratner Ranjay Krishna Chen-Yu Lee Tomas Pfister ALM 204 498 0 03 May 2023
Shapley Head Pruning: Identifying and Removing Interference in Multilingual Transformers William B. Held Diyi Yang VLM 30 5 0 11 Oct 2022
BERT-of-Theseus: Compressing BERT by Progressive Module Replacing Canwen Xu Wangchunshu Zhou Tao Ge Furu Wei Ming Zhou 213 196 0 07 Feb 2020