Rigging the Lottery: Making All Tickets Winners

25 November 2019

Papers citing "Rigging the Lottery: Making All Tickets Winners"

50 / 152 papers shown

Title
ESL-SNNs: An Evolutionary Structure Learning Strategy for Spiking Neural Networks Jiangrong Shen Qi Xu Jian K. Liu Yueming Wang Gang Pan Huajin Tang 30 42 0 06 Jun 2023
Learning Task-preferred Inference Routes for Gradient De-conflict in Multi-output DNNs Yi Sun Xin Xu Jian Li Xiaochang Hu Yifei Shi L. Zeng 39 2 0 31 May 2023
Adaptive Sparsity Level during Training for Efficient Time Series Forecasting with Transformers Zahra Atashgahi Mykola Pechenizkiy Raymond N. J. Veldhuis Decebal Constantin Mocanu AI4TS AI4CE 36 1 0 28 May 2023
Sparse Weight Averaging with Multiple Particles for Iterative Magnitude Pruning Moonseok Choi Hyungi Lee G. Nam Juho Lee 45 2 0 24 May 2023
Cuttlefish: Low-Rank Model Training without All the Tuning Hongyi Wang Saurabh Agarwal Pongsakorn U-chupala Yoshiki Tanaka Eric P. Xing Dimitris Papailiopoulos OffRL 63 22 0 04 May 2023
NTK-SAP: Improving neural network pruning by aligning training dynamics Yite Wang Dawei Li Ruoyu Sun 47 19 0 06 Apr 2023
Training Strategies for Vision Transformers for Object Detection Apoorv Singh 33 4 0 05 Apr 2023
Scaling Expert Language Models with Unsupervised Domain Discovery Suchin Gururangan Margaret Li M. Lewis Weijia Shi Tim Althoff Noah A. Smith Luke Zettlemoyer MoE 30 46 0 24 Mar 2023
Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency Vithursan Thangarasa Shreyas Saxena Abhay Gupta Sean Lie 41 3 0 21 Mar 2023
Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together! Shiwei Liu Tianlong Chen Zhenyu Zhang Xuxi Chen Tianjin Huang Ajay Jaiswal Zhangyang Wang 39 29 0 03 Mar 2023
Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detection Zhen Cheng Fei Zhu Xu-Yao Zhang Cheng-Lin Liu MoMe OODD 45 11 0 02 Mar 2023
Balanced Training for Sparse GANs Yite Wang Jing Wu N. Hovakimyan Ruoyu Sun 50 9 0 28 Feb 2023
Fast as CHITA: Neural Network Pruning with Combinatorial Optimization Riade Benbaki Wenyu Chen X. Meng Hussein Hazimeh Natalia Ponomareva Zhe Zhao Rahul Mazumder 21 26 0 28 Feb 2023
A Unified Framework for Soft Threshold Pruning Yanqing Chen Zhengyu Ma Wei Fang Xiawu Zheng Zhaofei Yu Yonghong Tian 88 19 0 25 Feb 2023
Bi-directional Masks for Efficient N:M Sparse Training Yuxin Zhang Yiting Luo Mingbao Lin Mingliang Xu Jingjing Xie Rongrong Ji Rongrong Ji 52 15 0 13 Feb 2023
Pruning Deep Neural Networks from a Sparsity Perspective Enmao Diao G. Wang Jiawei Zhan Yuhong Yang Jie Ding Vahid Tarokh 29 30 0 11 Feb 2023
SparseProp: Efficient Sparse Backpropagation for Faster Training of Neural Networks Mahdi Nikdan Tommaso Pegolotti Eugenia Iofinova Eldar Kurtic Dan Alistarh 26 11 0 09 Feb 2023
A Survey on Efficient Training of Transformers Bohan Zhuang Jing Liu Zizheng Pan Haoyu He Yuetian Weng Chunhua Shen 36 47 0 02 Feb 2023
An Empirical Study on the Transferability of Transformer Modules in Parameter-Efficient Fine-Tuning Mohammad AkbarTajari S. Rajaee Mohammad Taher Pilehvar 19 2 0 01 Feb 2023
Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning Huan Wang Can Qin Yue Bai Yun Fu 37 20 0 12 Jan 2023
Balance is Essence: Accelerating Sparse Training via Adaptive Gradient Correction Bowen Lei Dongkuan Xu Ruqi Zhang Shuren He Bani Mallick 44 6 0 09 Jan 2023
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot Elias Frantar Dan Alistarh VLM 41 643 0 02 Jan 2023
Dynamic Sparse Network for Time Series Classification: Learning What to "see'' Qiao Xiao Boqian Wu Yu Zhang Shiwei Liu Mykola Pechenizkiy Elena Mocanu Decebal Constantin Mocanu AI4TS 43 28 0 19 Dec 2022
Dynamic Sparse Training via Balancing the Exploration-Exploitation Trade-off Shaoyi Huang Bowen Lei Dongkuan Xu Hongwu Peng Yue Sun Mimi Xie Caiwen Ding 31 19 0 30 Nov 2022
You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets Tianjin Huang Tianlong Chen Meng Fang Vlado Menkovski Jiaxu Zhao ... Yulong Pei Decebal Constantin Mocanu Zhangyang Wang Mykola Pechenizkiy Shiwei Liu GNN 52 14 0 28 Nov 2022
PipeFisher: Efficient Training of Large Language Models Using Pipelining and Fisher Information Matrices Kazuki Osawa Shigang Li Torsten Hoefler AI4CE 35 24 0 25 Nov 2022
Exploiting the Partly Scratch-off Lottery Ticket for Quantization-Aware Training Mingliang Xu Gongrui Nan Yuxin Zhang Rongrong Ji Rongrong Ji MQ 23 3 0 12 Nov 2022
Efficient Traffic State Forecasting using Spatio-Temporal Network Dependencies: A Sparse Graph Neural Network Approach Bin Lei Shaoyi Huang Caiwen Ding Monika Filipovska GNN AI4TS 22 0 0 06 Nov 2022
Robust Lottery Tickets for Pre-trained Language Models Rui Zheng Rong Bao Yuhao Zhou Di Liang Sirui Wang Wei Wu Tao Gui Qi Zhang Xuanjing Huang AAML 34 13 0 06 Nov 2022
Gradient-based Weight Density Balancing for Robust Dynamic Sparse Training Mathias Parger Alexander Ertl Paul Eibensteiner J. H. Mueller Martin Winter M. Steinberger 34 0 0 25 Oct 2022
Parameter-Efficient Masking Networks Yue Bai Huan Wang Xu Ma Yitian Zhang Zhiqiang Tao Yun Fu 31 10 0 13 Oct 2022
Clustering the Sketch: A Novel Approach to Embedding Table Compression Henry Ling-Hei Tsang Thomas Dybdahl Ahle 45 1 0 12 Oct 2022
Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approach Peng Mi Li Shen Tianhe Ren Yiyi Zhou Xiaoshuai Sun Rongrong Ji Dacheng Tao AAML 43 69 0 11 Oct 2022
SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapters Shwai He Liang Ding Daize Dong Miao Zhang Dacheng Tao MoE 37 87 0 09 Oct 2022
Advancing Model Pruning via Bi-level Optimization Yihua Zhang Yuguang Yao Parikshit Ram Pu Zhao Tianlong Chen Min-Fong Hong Yanzhi Wang Sijia Liu 56 68 0 08 Oct 2022
EPIC TTS Models: Empirical Pruning Investigations Characterizing Text-To-Speech Models Perry Lam Huayun Zhang Nancy F. Chen Berrak Sisman 21 2 0 22 Sep 2022
Towards Sparsification of Graph Neural Networks Hongwu Peng Deniz Gurevin Shaoyi Huang Tong Geng Weiwen Jiang O. Khan Caiwen Ding GNN 30 24 0 11 Sep 2022
Safety and Performance, Why not Both? Bi-Objective Optimized Model Compression toward AI Software Deployment Jie Zhu Leye Wang Xiao Han 35 9 0 11 Aug 2022
Green, Quantized Federated Learning over Wireless Networks: An Energy-Efficient Design Minsu Kim Walid Saad Mohammad Mozaffari Merouane Debbah FedML MQ 36 28 0 19 Jul 2022
Comprehensive Graph Gradual Pruning for Sparse Training in Graph Neural Networks Chuang Liu Xueqi Ma Yinbing Zhan Liang Ding Dapeng Tao Bo Du Wenbin Hu Danilo Mandic 44 29 0 18 Jul 2022
SInGE: Sparsity via Integrated Gradients Estimation of Neuron Relevance Edouard Yvinec Arnaud Dapogny Matthieu Cord Kévin Bailly 47 9 0 08 Jul 2022
DRESS: Dynamic REal-time Sparse Subnets Zhongnan Qu Syed Shakib Sarwar Xin Dong Yuecheng Li Huseyin Ekin Sumbul B. D. Salvo 3DH 23 1 0 01 Jul 2022
Winning the Lottery Ahead of Time: Efficient Early Network Pruning John Rachwan Daniel Zügner Bertrand Charpentier Simon Geisler Morgane Ayle Stephan Günnemann 32 24 0 21 Jun 2022
Spartan: Differentiable Sparsity via Regularized Transportation Kai Sheng Tai Taipeng Tian Ser-Nam Lim 34 11 0 27 May 2022
Compression-aware Training of Neural Networks using Frank-Wolfe Max Zimmer Christoph Spiegel Sebastian Pokutta 31 9 0 24 May 2022
Convolutional and Residual Networks Provably Contain Lottery Tickets R. Burkholz UQCV MLT 42 13 0 04 May 2022
Most Activation Functions Can Win the Lottery Without Excessive Depth R. Burkholz MLT 79 18 0 04 May 2022
LilNetX: Lightweight Networks with EXtreme Model Compression and Structured Sparsification Sharath Girish Kamal Gupta Saurabh Singh Abhinav Shrivastava 40 11 0 06 Apr 2022
Dynamic Focus-aware Positional Queries for Semantic Segmentation Haoyu He Jianfei Cai Zizheng Pan Jing Liu Jing Zhang Dacheng Tao Bohan Zhuang 34 17 0 04 Apr 2022
Automated Progressive Learning for Efficient Training of Vision Transformers Changlin Li Bohan Zhuang Guangrun Wang Xiaodan Liang Xiaojun Chang Yi Yang 33 46 0 28 Mar 2022