SparseProp: Efficient Sparse Backpropagation for Faster Training of
Neural Networks

SparseProp: Efficient Sparse Backpropagation for Faster Training of Neural Networks

9 February 2023

Tommaso Pegolotti

Eugenia Iofinova

Dan Alistarh

Papers citing "SparseProp: Efficient Sparse Backpropagation for Faster Training of Neural Networks"

15 / 15 papers shown

Title
An Efficient Row-Based Sparse Fine-Tuning Cen-Jhih Li Aditya Bhaskara 52 0 0 17 Feb 2025
Advancing On-Device Neural Network Training with TinyPropv2: Dynamic, Sparse, and Efficient Backpropagation Marcus Rüb Axel Sikora Daniel Mueller-Gritschneder 30 1 0 11 Sep 2024
SAFT: Towards Out-of-Distribution Generalization in Fine-Tuning Bac Nguyen Stefan Uhlich Fabien Cardinaux Lukas Mauch Marzieh Edraki Aaron Courville OODD CLL VLM 52 3 0 03 Jul 2024
TriLoRA: Integrating SVD for Advanced Style Personalization in Text-to-Image Generation Chengcheng Feng Mu He Qiuyu Tian Haojie Yin Xiaofang Zhao Hongwei Tang Xingqiang Wei DiffM 30 3 0 18 May 2024
Efficient Backpropagation with Variance-Controlled Adaptive Sampling Ziteng Wang Jianfei Chen Jun Zhu BDL 32 2 0 27 Feb 2024
RoSA: Accurate Parameter-Efficient Fine-Tuning via Robust Adaptation Mahdi Nikdan Soroush Tabesh Elvir Crnčević Dan Alistarh 8 27 0 09 Jan 2024
Accurate Neural Network Pruning Requires Rethinking Sparse Optimization Denis Kuznedelev Eldar Kurtic Eugenia Iofinova Elias Frantar Alexandra Peste Dan Alistarh VLM 21 11 0 03 Aug 2023
Dynamic Sparse Training with Structured Sparsity Mike Lasby A. Golubeva Utku Evci Mihai Nica Yani Andrew Ioannou 29 19 0 03 May 2023
Beyond Multilayer Perceptrons: Investigating Complex Topologies in Neural Networks T. Boccato Matteo Ferrante A. Duggento N. Toschi 28 2 0 31 Mar 2023
CrAM: A Compression-Aware Minimizer Alexandra Peste Adrian Vladu Eldar Kurtic Christoph H. Lampert Dan Alistarh 24 8 0 28 Jul 2022
Powerpropagation: A sparsity inducing weight reparameterisation Jonathan Richard Schwarz Siddhant M. Jayakumar Razvan Pascanu P. Latham Yee Whye Teh 87 54 0 01 Oct 2021
Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks Itay Hubara Brian Chmiel Moshe Island Ron Banner S. Naor Daniel Soudry 44 110 0 16 Feb 2021
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks Torsten Hoefler Dan Alistarh Tal Ben-Nun Nikoli Dryden Alexandra Peste MQ 139 684 0 31 Jan 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,950 0 20 Apr 2018
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 284 39,190 0 01 Sep 2014