Winning the Lottery with Continuous Sparsification

10 December 2019

Papers citing "Winning the Lottery with Continuous Sparsification"

35 / 35 papers shown

Title
Bigram Subnetworks: Mapping to Next Tokens in Transformer Language Models Tyler A. Chang Benjamin Bergen 58 0 0 21 Apr 2025
Deep Weight Factorization: Sparse Learning Through the Lens of Artificial Symmetries Chris Kolb T. Weber Bernd Bischl David Rügamer 120 0 0 04 Feb 2025
Playing the Lottery With Concave Regularizers for Sparse Trainable Neural Networks Giulia Fracastoro Sophie M. Fosson Andrea Migliorati G. Calafiore 50 1 0 19 Jan 2025
Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Models Philipp Mondorf Sondre Wold Yun Xue 46 0 0 02 Oct 2024
Mask in the Mirror: Implicit Sparsification Tom Jacobs R. Burkholz 52 3 0 19 Aug 2024
SequentialAttention++ for Block Sparsification: Differentiable Pruning Meets Combinatorial Optimization T. Yasuda Kyriakos Axiotis Gang Fu M. Bateni Vahab Mirrokni 49 0 0 27 Feb 2024
Uncovering Intermediate Variables in Transformers using Circuit Probing Michael A. Lepori Thomas Serre Ellie Pavlick 78 7 0 07 Nov 2023
NTK-SAP: Improving neural network pruning by aligning training dynamics Yite Wang Dawei Li Ruoyu Sun 47 19 0 06 Apr 2023
On Model Compression for Neural Networks: Framework, Algorithm, and Convergence Guarantee Chenyang Li Jihoon Chung Mengnan Du Haimin Wang Xianlian Zhou Bohao Shen 33 1 0 13 Mar 2023
Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together! Shiwei Liu Tianlong Chen Zhenyu Zhang Xuxi Chen Tianjin Huang Ajay Jaiswal Zhangyang Wang 39 29 0 03 Mar 2023
Balanced Training for Sparse GANs Yite Wang Jing Wu N. Hovakimyan Ruoyu Sun 53 9 0 28 Feb 2023
A Unified Framework for Soft Threshold Pruning Yanqing Chen Zhengyu Ma Wei Fang Xiawu Zheng Zhaofei Yu Yonghong Tian 88 19 0 25 Feb 2023
CSQ: Growing Mixed-Precision Quantization Scheme with Bi-level Continuous Sparsification Lirui Xiao Huanrui Yang Zhen Dong Kurt Keutzer Li Du Shanghang Zhang MQ 29 10 0 06 Dec 2022
Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? Mansheej Paul F. Chen Brett W. Larsen Jonathan Frankle Surya Ganguli Gintare Karolina Dziugaite UQCV 47 38 0 06 Oct 2022
Optimizing Connectivity through Network Gradients for Restricted Boltzmann Machines A. C. N. D. Oliveira Daniel R. Figueiredo 32 0 0 14 Sep 2022
Spartan: Differentiable Sparsity via Regularized Transportation Kai Sheng Tai Taipeng Tian Ser-Nam Lim 34 11 0 27 May 2022
Convolutional and Residual Networks Provably Contain Lottery Tickets R. Burkholz UQCV MLT 42 13 0 04 May 2022
Most Activation Functions Can Win the Lottery Without Excessive Depth R. Burkholz MLT 79 18 0 04 May 2022
LilNetX: Lightweight Networks with EXtreme Model Compression and Structured Sparsification Sharath Girish Kamal Gupta Saurabh Singh Abhinav Shrivastava 40 11 0 06 Apr 2022
Playing Lottery Tickets in Style Transfer Models Meihao Kong Jing Huo Wenbin Li Jing Wu Yu-Kun Lai Yang Gao 29 1 0 25 Mar 2022
Rare Gems: Finding Lottery Tickets at Initialization Kartik K. Sreenivasan Jy-yong Sohn Liu Yang Matthew Grinde Alliot Nagle Hongyi Wang Eric P. Xing Kangwook Lee Dimitris Papailiopoulos 32 42 0 24 Feb 2022
Deadwooding: Robust Global Pruning for Deep Neural Networks Sawinder Kaur Ferdinando Fioretto Asif Salekin 32 4 0 10 Feb 2022
Efficient Neural Network Training via Forward and Backward Propagation Sparsification Xiao Zhou Weizhong Zhang Zonghao Chen Shizhe Diao Tong Zhang 40 46 0 10 Nov 2021
Drawing Robust Scratch Tickets: Subnetworks with Inborn Robustness Are Found within Randomly Initialized Networks Yonggan Fu Qixuan Yu Yang Zhang Shan-Hung Wu Ouyang Xu David D. Cox Yingyan Lin AAML OOD 33 29 0 26 Oct 2021
Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learning Soufiane Hayou Bo He Gintare Karolina Dziugaite 37 2 0 22 Oct 2021
Lottery Tickets with Nonzero Biases Jonas Fischer Advait Gadhikar R. Burkholz 27 6 0 21 Oct 2021
An Information Theory-inspired Strategy for Automatic Network Pruning Xiawu Zheng Yuexiao Ma Teng Xi Gang Zhang Errui Ding Yuchao Li Jie Chen Yonghong Tian Rongrong Ji 54 13 0 19 Aug 2021
Sparse Training via Boosting Pruning Plasticity with Neuroregeneration Shiwei Liu Tianlong Chen Xiaohan Chen Zahra Atashgahi Lu Yin Huanyu Kou Li Shen Mykola Pechenizkiy Zhangyang Wang Decebal Constantin Mocanu 45 112 0 19 Jun 2021
Towards Understanding Iterative Magnitude Pruning: Why Lottery Tickets Win Jaron Maene Mingxiao Li Marie-Francine Moens 15 15 0 13 Jun 2021
GANs Can Play Lottery Tickets Too Xuxi Chen Zhenyu Zhang Yongduo Sui Tianlong Chen GAN 24 58 0 31 May 2021
Playing Lottery Tickets with Vision and Language Zhe Gan Yen-Chun Chen Linjie Li Tianlong Chen Yu Cheng Shuohang Wang Jingjing Liu Lijuan Wang Zicheng Liu VLM 111 54 0 23 Apr 2021
Lottery Jackpots Exist in Pre-trained Models Yuxin Zhang Mingbao Lin Yan Wang Rongrong Ji Rongrong Ji 35 15 0 18 Apr 2021
The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models Tianlong Chen Jonathan Frankle Shiyu Chang Sijia Liu Yang Zhang Michael Carbin Zhangyang Wang 27 123 0 12 Dec 2020
Linear Mode Connectivity and the Lottery Ticket Hypothesis Jonathan Frankle Gintare Karolina Dziugaite Daniel M. Roy Michael Carbin MoMe 43 601 0 11 Dec 2019
A Brain-inspired Algorithm for Training Highly Sparse Neural Networks Zahra Atashgahi Joost Pieterse Shiwei Liu Decebal Constantin Mocanu Raymond N. J. Veldhuis Mykola Pechenizkiy 35 15 0 17 Mar 2019