How Well Do Sparse Imagenet Models Transfer?

26 November 2021

Dan Alistarh

Papers citing "How Well Do Sparse Imagenet Models Transfer?"

39 / 39 papers shown

Title
Token Coordinated Prompt Attention is Needed for Visual Prompting Zichen Liu Xu Zou Gang Hua Jiahuan Zhou 34 0 0 05 May 2025
Visual Variational Autoencoder Prompt Tuning Xi Xiao Yunbei Zhang Y. Li Xingjian Li Tianyang Wang Jihun Hamm X. Wang Min Xu VPVLM VLM 58 0 0 22 Mar 2025
Visual Fourier Prompt Tuning Runjia Zeng Cheng Han Qifan Wang Chunshu Wu Tong Geng Lifu Huang Ying Nian Wu Dongfang Liu VPVLM VLM 48 6 0 02 Nov 2024
Inducing Semi-Structured Sparsity by Masking for Efficient Model Inference in Convolutional Networks David A. Danhofer 27 0 0 01 Nov 2024
AdaptGCD: Multi-Expert Adapter Tuning for Generalized Category Discovery Yuxun Qu Yongqiang Tang Chenyang Zhang Wensheng Zhang 26 0 0 29 Oct 2024
Revisiting Prefix-tuning: Statistical Benefits of Reparameterization among Prompts Minh Le Chau Nguyen Huy Nguyen Quyen Tran Trung Le Nhat Ho 32 4 0 03 Oct 2024
3D Point Cloud Network Pruning: When Some Weights Do not Matter Amrijit Biswas M. Hossain M. M. L. Elahi A. Cheraghian Fuad Rahman Nabeel Mohammed Shafin Rahman 3DPC 20 1 0 26 Aug 2024
FIARSE: Model-Heterogeneous Federated Learning via Importance-Aware Submodel Extraction Feijie Wu Xingchen Wang Yaqing Wang Tianci Liu Lu Su Jing Gao FedML 43 3 0 28 Jul 2024
Which Backbone to Use: A Resource-efficient Domain Specific Comparison for Computer Vision Pranav Jeevan Amit Sethi VLM 31 4 0 09 Jun 2024
Finding Lottery Tickets in Vision Models via Data-driven Spectral Foresight Pruning Leonardo Iurada Marco Ciccone Tatiana Tommasi 36 3 0 03 Jun 2024
Multilingual Brain Surgeon: Large Language Models Can be Compressed Leaving No Language Behind Hongchuan Zeng Hongshen Xu Lu Chen Kai Yu 47 5 0 06 Apr 2024
A Survey of Lottery Ticket Hypothesis Bohan Liu Zijie Zhang Peixiong He Zhensen Wang Yang Xiao Ruimeng Ye Yang Zhou Wei-Shinn Ku Bo Hui UQCV 37 12 0 07 Mar 2024
Facing the Elephant in the Room: Visual Prompt Tuning or Full Finetuning? Cheng Han Qifan Wang Yiming Cui Wenguan Wang Lifu Huang Siyuan Qi Dongfang Liu VLM 44 19 0 23 Jan 2024
Robustness to distribution shifts of compressed networks for edge devices Lulan Shen Ali Edalati Brett H. Meyer Warren Gross James J. Clark 25 0 0 22 Jan 2024
Accelerating Deep Neural Networks via Semi-Structured Activation Sparsity Matteo Grimaldi Darshan C. Ganji Ivan Lazarevich Sudhakar Sah 6 10 0 12 Sep 2023
Uncovering the Hidden Cost of Model Compression Diganta Misra Muawiz Chaudhary Agam Goyal Bharat Runwal Pin-Yu Chen VLM 24 0 0 29 Aug 2023
Accurate Neural Network Pruning Requires Rethinking Sparse Optimization Denis Kuznedelev Eldar Kurtic Eugenia Iofinova Elias Frantar Alexandra Peste Dan Alistarh VLM 21 11 0 03 Aug 2023
E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuning Cheng Han Qifan Wang Yiming Cui Zhiwen Cao Wenguan Wang Siyuan Qi Dongfang Liu VPVLM VLM 17 46 0 25 Jul 2023
A Survey of Techniques for Optimizing Transformer Inference Krishna Teja Chitty-Venkata Sparsh Mittal M. Emani V. Vishwanath Arun Somani 31 62 0 16 Jul 2023
Dynamic Sparsity Is Channel-Level Sparsity Learner Lu Yin Gen Li Meng Fang Lijuan Shen Tianjin Huang Zhangyang Wang Vlado Menkovski Xiaolong Ma Mykola Pechenizkiy Shiwei Liu 17 20 0 30 May 2023
Knowledge Distillation Performs Partial Variance Reduction M. Safaryan Alexandra Peste Dan Alistarh 6 6 0 27 May 2023
Dynamic Sparse Training with Structured Sparsity Mike Lasby A. Golubeva Utku Evci Mihai Nica Yani Andrew Ioannou 29 19 0 03 May 2023
Bias in Pruned Vision Models: In-Depth Analysis and Countermeasures Eugenia Iofinova Alexandra Peste Dan Alistarh 18 9 0 25 Apr 2023
Robust Tickets Can Transfer Better: Drawing More Transferable Subnetworks in Transfer Learning Y. Fu Ye Yuan Shang Wu Jiayi Yuan Yingyan Lin OOD 50 3 0 24 Apr 2023
oBERTa: Improving Sparse Transfer Learning via improved initialization, distillation, and pruning regimes Daniel Fernando Campos Alexandre Marques Mark Kurtz Chengxiang Zhai VLM AAML 11 2 0 30 Mar 2023
Vision Models Can Be Efficiently Specialized via Few-Shot Task-Aware Compression Denis Kuznedelev Soroush Tabesh Kimia Noorbakhsh Elias Frantar Sara Beery Eldar Kurtic Dan Alistarh MQ VLM 21 2 0 25 Mar 2023
Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency Vithursan Thangarasa Shreyas Saxena Abhay Gupta Sean Lie 21 3 0 21 Mar 2023
Towards Optimal Compression: Joint Pruning and Quantization Ben Zandonati Glenn Bucagu Adrian Alan Pol M. Pierini Olya Sirkin Tal Kopetz MQ 17 2 0 15 Feb 2023
SparseProp: Efficient Sparse Backpropagation for Faster Training of Neural Networks Mahdi Nikdan Tommaso Pegolotti Eugenia Iofinova Eldar Kurtic Dan Alistarh 12 11 0 09 Feb 2023
Ten Lessons We Have Learned in the New "Sparseland": A Short Handbook for Sparse Neural Network Researchers Shiwei Liu Zhangyang Wang 27 30 0 06 Feb 2023
GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers Elias Frantar Saleh Ashkboos Torsten Hoefler Dan Alistarh MQ 14 882 0 31 Oct 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 28 109 0 31 Aug 2022
CrAM: A Compression-Aware Minimizer Alexandra Peste Adrian Vladu Eldar Kurtic Christoph H. Lampert Dan Alistarh 24 8 0 28 Jul 2022
Federated Progressive Sparsification (Purge, Merge, Tune)+ Dimitris Stripelis Umang Gupta Greg Ver Steeg J. Ambite FedML 13 9 0 26 Apr 2022
SD-Conv: Towards the Parameter-Efficiency of Dynamic Convolution Shwai He Chenbo Jiang Daize Dong Liang Ding 28 5 0 05 Apr 2022
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks Torsten Hoefler Dan Alistarh Tal Ben-Nun Nikoli Dryden Alexandra Peste MQ 139 684 0 31 Jan 2021
The Lottery Ticket Hypothesis for Object Recognition Sharath Girish Shishira R. Maiya Kamal Gupta Hao Chen L. Davis Abhinav Shrivastava 75 60 0 08 Dec 2020
The Lottery Ticket Hypothesis for Pre-trained BERT Networks Tianlong Chen Jonathan Frankle Shiyu Chang Sijia Liu Yang Zhang Zhangyang Wang Michael Carbin 148 345 0 23 Jul 2020
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 948 20,549 0 17 Apr 2017