Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging

Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging

29 June 2023

Christoph Spiegel

Papers citing "Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging"

12 / 12 papers shown

Title
On the Byzantine-Resilience of Distillation-Based Federated Learning Christophe Roux Max Zimmer S. Pokutta AAML 49 1 0 19 Feb 2024
Instant Soup: Cheap Pruning Ensembles in A Single Pass Can Draw Lottery Tickets from Large Models A. Jaiswal Shiwei Liu Tianlong Chen Ying Ding Zhangyang Wang VLM 32 22 0 18 Jun 2023
PopulAtion Parameter Averaging (PAPA) Alexia Jolicoeur-Martineau Emy Gervais Kilian Fatras Yan Zhang Simon Lacoste-Julien MoMe 40 17 0 06 Apr 2023
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 239 312 0 11 Sep 2022
Superposing Many Tickets into One: A Performance Booster for Sparse Neural Network Training Lu Yin Vlado Menkovski Meng Fang Tianjin Huang Yulong Pei Mykola Pechenizkiy D. Mocanu Shiwei Liu 24 8 0 30 May 2022
Pruning has a disparate impact on model accuracy Cuong Tran Ferdinando Fioretto Jung-Eun Kim Rakshit Naidu 32 38 0 26 May 2022
Diverse Weight Averaging for Out-of-Distribution Generalization Alexandre Ramé Matthieu Kirchmeyer Thibaud Rahier A. Rakotomamonjy Patrick Gallinari Matthieu Cord OOD 188 128 0 19 May 2022
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks Torsten Hoefler Dan Alistarh Tal Ben-Nun Nikoli Dryden Alexandra Peste MQ 139 684 0 31 Jan 2021
What is the State of Neural Network Pruning? Davis W. Blalock Jose Javier Gonzalez Ortiz Jonathan Frankle John Guttag 178 1,027 0 06 Mar 2020
Comparing Rewinding and Fine-tuning in Neural Network Pruning Alex Renda Jonathan Frankle Michael Carbin 222 382 0 05 Mar 2020
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles Balaji Lakshminarayanan Alexander Pritzel Charles Blundell UQCV BDL 268 5,652 0 05 Dec 2016
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 282 39,170 0 01 Sep 2014