Learning Sparse Neural Networks through $L_0$ Regularization

4 December 2017

Papers citing "Learning Sparse Neural Networks through $L_0$ Regularization"

50 / 154 papers shown

Title
LOFT: Finding Lottery Tickets through Filter-wise Training Qihan Wang Chen Dun Fangshuo Liao C. Jermaine Anastasios Kyrillidis 18 3 0 28 Oct 2022
OLLA: Optimizing the Lifetime and Location of Arrays to Reduce the Memory Usage of Neural Networks Benoit Steiner Mostafa Elhoushi Jacob Kahn James Hegarty 29 8 0 24 Oct 2022
Pruning by Active Attention Manipulation Z. Babaiee Lucas Liebenwein Ramin Hasani Daniela Rus Radu Grosu 17 0 0 20 Oct 2022
Packed-Ensembles for Efficient Uncertainty Estimation Olivier Laurent Adrien Lafage Enzo Tartaglione Geoffrey Daniel Jean-Marc Martinez Andrei Bursuc Gianni Franchi OODD 44 32 0 17 Oct 2022
The Open-World Lottery Ticket Hypothesis for OOD Intent Classification Yunhua Zhou Pengyu Wang Peiju Liu Yuxin Wang Xipeng Qiu 21 2 0 13 Oct 2022
Sequential Attention for Feature Selection T. Yasuda M. Bateni Lin Chen Matthew Fahrbach Gang Fu Vahab Mirrokni 29 11 0 29 Sep 2022
Make Heterophily Graphs Better Fit GNN: A Graph Rewiring Approach Wendong Bi Lun Du Qiang Fu Yanlin Wang Shi Han Dongmei Zhang 22 27 0 17 Sep 2022
The Role Of Biology In Deep Learning Robert Bain 22 0 0 07 Sep 2022
What to Prune and What Not to Prune at Initialization Maha Haroon 13 0 0 06 Sep 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 28 109 0 31 Aug 2022
Mixed-Precision Neural Networks: A Survey M. Rakka M. Fouda Pramod P. Khargonekar Fadi J. Kurdahi MQ 18 11 0 11 Aug 2022
SBPF: Sensitiveness Based Pruning Framework For Convolutional Neural Network On Image Classification Yihe Lu Maoguo Gong Wei Zhao Kaiyuan Feng Hao Li VLM 29 0 0 09 Aug 2022
Doge Tickets: Uncovering Domain-general Language Models by Playing Lottery Tickets Yi Yang Chen Zhang Benyou Wang Dawei Song LRM 18 6 0 20 Jul 2022
Probing via Prompting Jiaoda Li Ryan Cotterell Mrinmaya Sachan 29 13 0 04 Jul 2022
Detecting Arbitrary Order Beneficial Feature Interactions for Recommender Systems Yixin Su Yunxiang Zhao S. Erfani Junhao Gan Rui Zhang 55 24 0 28 Jun 2022
Sparse Double Descent: Where Network Pruning Aggravates Overfitting Zhengqi He Zeke Xie Quanzhi Zhu Zengchang Qin 69 27 0 17 Jun 2022
Spartan: Differentiable Sparsity via Regularized Transportation Kai Sheng Tai Taipeng Tian Ser-Nam Lim 17 11 0 27 May 2022
PointDistiller: Structured Knowledge Distillation Towards Efficient and Compact 3D Detection Linfeng Zhang Runpei Dong Hung-Shuo Tai Kaisheng Ma 3DPC 72 46 0 23 May 2022
AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Tasks Chin-Lun Fu Zih-Ching Chen Yun-Ru Lee Hung-yi Lee 28 44 0 30 Apr 2022
RAPQ: Rescuing Accuracy for Power-of-Two Low-bit Post-training Quantization Hongyi Yao Pu Li Jian Cao Xiangcheng Liu Chenying Xie Bin Wang MQ 19 12 0 26 Apr 2022
Merging of neural networks Martin Pasen Vladimír Boza FedML MoMe 30 2 0 21 Apr 2022
Accelerating Attention through Gradient-Based Learned Runtime Pruning Zheng Li Soroush Ghodrati Amir Yazdanbakhsh H. Esmaeilzadeh Mingu Kang 19 16 0 07 Apr 2022
LilNetX: Lightweight Networks with EXtreme Model Compression and Structured Sparsification Sharath Girish Kamal Gupta Saurabh Singh Abhinav Shrivastava 28 11 0 06 Apr 2022
Explainability in Graph Neural Networks: An Experimental Survey Peibo Li Yixing Yang M. Pagnucco Yang Song 21 31 0 17 Mar 2022
Lightweight Jet Reconstruction and Identification as an Object Detection Task Adrian Alan Pol T. Aarrestad E. Govorkova Roi Halily Anat Klempner ... Vladimir Loncar J. Ngadiuba M. Pierini Olya Sirkin S. Summers 19 2 0 09 Feb 2022
Maximizing information from chemical engineering data sets: Applications to machine learning Alexander Thebelt Johannes Wiebe Jan Kronqvist Calvin Tsay Ruth Misener AI4CE 34 68 0 25 Jan 2022
Pruning-aware Sparse Regularization for Network Pruning Nanfei Jiang Xu Zhao Chaoyang Zhao Yongqi An Ming Tang Jinqiao Wang 3DPC 13 12 0 18 Jan 2022
Transferability in Deep Learning: A Survey Junguang Jiang Yang Shu Jianmin Wang Mingsheng Long OOD 31 101 0 15 Jan 2022
Recursive Least Squares for Training and Pruning Convolutional Neural Networks Tianzong Yu Chunyuan Zhang Yuan Wang Meng-tao Ma Qingwei Song 22 1 0 13 Jan 2022
Automatic Mixed-Precision Quantization Search of BERT Changsheng Zhao Ting Hua Yilin Shen Qian Lou Hongxia Jin MQ 17 19 0 30 Dec 2021
Compact Multi-level Sparse Neural Networks with Input Independent Dynamic Rerouting Minghai Qin Tianyun Zhang Fei Sun Yen-kuang Chen M. Fardad Yanzhi Wang Yuan Xie 31 0 0 21 Dec 2021
From Dense to Sparse: Contrastive Pruning for Better Pre-trained Language Model Compression Runxin Xu Fuli Luo Chengyu Wang Baobao Chang Jun Huang Songfang Huang Fei Huang VLM 27 25 0 14 Dec 2021
Sparse Interventions in Language Models with Differentiable Masking Nicola De Cao Leon Schmid Dieuwke Hupkes Ivan Titov 32 27 0 13 Dec 2021
Training Neural Networks with Fixed Sparse Masks Yi-Lin Sung Varun Nair Colin Raffel FedML 18 196 0 18 Nov 2021
Efficient Neural Network Training via Forward and Backward Propagation Sparsification Xiao Zhou Weizhong Zhang Zonghao Chen Shizhe Diao Tong Zhang 26 46 0 10 Nov 2021
Support Recovery with Stochastic Gates: Theory and Application for Linear Models Soham Jana Henry Li Yutaro Yamada Ofir Lindenbaum 23 5 0 29 Oct 2021
Sparsely Changing Latent States for Prediction and Planning in Partially Observable Domains Christian Gumbsch Martin Volker Butz Georg Martius AI4CE 18 21 0 29 Oct 2021
Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learning Soufiane Hayou Bo He Gintare Karolina Dziugaite 20 2 0 22 Oct 2021
End-to-End Supermask Pruning: Learning to Prune Image Captioning Models J. Tan C. Chan Joon Huang Chuah VLM 49 16 0 07 Oct 2021
A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning Iris A. M. Huijben W. Kool Max B. Paulus Ruud J. G. van Sloun 26 93 0 04 Oct 2021
Powerpropagation: A sparsity inducing weight reparameterisation Jonathan Richard Schwarz Siddhant M. Jayakumar Razvan Pascanu P. Latham Yee Whye Teh 87 54 0 01 Oct 2021
The Stem Cell Hypothesis: Dilemma behind Multi-Task Learning with Transformer Encoders Han He Jinho D. Choi 43 87 0 14 Sep 2021
Survey of Low-Resource Machine Translation Barry Haddow Rachel Bawden Antonio Valerio Miceli Barone Jindvrich Helcl Alexandra Birch AIMat 31 147 0 01 Sep 2021
Layer-wise Model Pruning based on Mutual Information Chun Fan Jiwei Li Xiang Ao Fei Wu Yuxian Meng Xiaofei Sun 38 19 0 28 Aug 2021
Differentiable Subset Pruning of Transformer Heads Jiaoda Li Ryan Cotterell Mrinmaya Sachan 37 53 0 10 Aug 2021
Learning a Neural Diff for Speech Models J. Macoskey Grant P. Strimel Ariya Rastrow 13 2 0 03 Aug 2021
Sparse Training via Boosting Pruning Plasticity with Neuroregeneration Shiwei Liu Tianlong Chen Xiaohan Chen Zahra Atashgahi Lu Yin Huanyu Kou Li Shen Mykola Pechenizkiy Zhangyang Wang D. Mocanu 34 111 0 19 Jun 2021
Can Subnetwork Structure be the Key to Out-of-Distribution Generalization? Dinghuai Zhang Kartik Ahuja Yilun Xu Yisen Wang Aaron Courville OOD 20 95 0 05 Jun 2021
Effective Sparsification of Neural Networks with Global Sparsity Constraint Xiao Zhou Weizhong Zhang Hang Xu Tong Zhang 19 61 0 03 May 2021
Do All MobileNets Quantize Poorly? Gaining Insights into the Effect of Quantization on Depthwise Separable Convolutional Networks Through the Eyes of Multi-scale Distributional Dynamics S. Yun Alexander Wong MQ 19 25 0 24 Apr 2021