Balance is Essence: Accelerating Sparse Training via Adaptive Gradient
Correction

Balance is Essence: Accelerating Sparse Training via Adaptive Gradient Correction

9 January 2023

Dongkuan Xu

Papers citing "Balance is Essence: Accelerating Sparse Training via Adaptive Gradient Correction"

8 / 8 papers shown

Title
Embracing Unknown Step by Step: Towards Reliable Sparse Training in Real World Bowen Lei Dongkuan Xu Ruqi Zhang Bani Mallick UQCV 26 0 0 29 Mar 2024
AutoReP: Automatic ReLU Replacement for Fast Private Network Inference Hongwu Peng Shaoyi Huang Tong Zhou Yukui Luo Chenghong Wang ... Tony Geng Kaleel Mahmood Wujie Wen Xiaolin Xu Caiwen Ding OffRL 32 38 0 20 Aug 2023
Powerpropagation: A sparsity inducing weight reparameterisation Jonathan Richard Schwarz Siddhant M. Jayakumar Razvan Pascanu P. Latham Yee Whye Teh 84 54 0 01 Oct 2021
Carbon Emissions and Large Neural Network Training David A. Patterson Joseph E. Gonzalez Quoc V. Le Chen Liang Lluís-Miquel Munguía D. Rothchild David R. So Maud Texier J. Dean AI4CE 236 626 0 21 Apr 2021
Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks Itay Hubara Brian Chmiel Moshe Island Ron Banner S. Naor Daniel Soudry 44 89 0 16 Feb 2021
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks Torsten Hoefler Dan Alistarh Tal Ben-Nun Nikoli Dryden Alexandra Peste MQ 128 679 0 31 Jan 2021
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 279 39,083 0 01 Sep 2014
A Proximal Stochastic Gradient Method with Progressive Variance Reduction Lin Xiao Tong Zhang ODL 76 736 0 19 Mar 2014