Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Science

15 July 2017

Decebal Constantin Mocanu

Papers citing "Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Science"

24 / 24 papers shown

Title
Efficient Shapley Value-based Non-Uniform Pruning of Large Language Models Chuan Sun Han Yu Lizhen Cui Xiaoxiao Li 320 2 0 03 May 2025
Sparse-to-Sparse Training of Diffusion Models Inês Cardoso Oliveira Decebal Constantin Mocanu Luis A. Leiva DiffM 116 0 0 30 Apr 2025
E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentation Boqian Wu Q. Xiao Shiwei Liu Lu Yin Mykola Pechenizkiy Decebal Constantin Mocanu M. V. Keulen Elena Mocanu MedIm 94 5 0 20 Feb 2025
Brain network science modelling of sparse neural networks enables Transformers and LLMs to perform as fully connected Yingtao Zhang Diego Cerretti Jialin Zhao Wenjing Wu Ziheng Liao Umberto Michieli C. Cannistraci 90 1 0 31 Jan 2025
SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training Tianjin Huang Ziquan Zhu Gaojie Jin Lu Liu Zhangyang Wang Shiwei Liu 77 3 0 12 Jan 2025
Pushing the Limits of Sparsity: A Bag of Tricks for Extreme Pruning Andy Li A. Durrant Milan Markovic Lu Yin Georgios Leontidis Tianlong Chen Lu Yin Georgios Leontidis 121 0 0 20 Nov 2024
Zeroth-Order Adaptive Neuron Alignment Based Pruning without Re-Training Elia Cunegatti Leonardo Lucio Custode Giovanni Iacca 84 0 0 11 Nov 2024
Layer-Adaptive State Pruning for Deep State Space Models Minseon Gwak Seongrok Moon Joohwan Ko PooGyeon Park 80 0 0 05 Nov 2024
Navigating Extremes: Dynamic Sparsity in Large Output Spaces Nasib Ullah Erik Schultheis Mike Lasby Yani Andrew Ioannou Rohit Babbar 52 0 0 05 Nov 2024
Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustness Boqian Wu Q. Xiao Shunxin Wang N. Strisciuglio Mykola Pechenizkiy M. V. Keulen Decebal Constantin Mocanu Elena Mocanu OOD 3DH 137 2 0 03 Oct 2024
Always-Sparse Training by Growing Connections with Guided Stochastic Exploration Mike Heddes Narayan Srinivasa T. Givargis Alexandru Nicolau 162 0 0 12 Jan 2024
PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs Max Zimmer Megi Andoni Christoph Spiegel Sebastian Pokutta VLM 86 10 0 23 Dec 2023
Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning F. Such Vashisht Madhavan Edoardo Conti Joel Lehman Kenneth O. Stanley Jeff Clune 70 690 0 18 Dec 2017
Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms Han Xiao Kashif Rasul Roland Vollgraf 173 8,807 0 25 Aug 2017
Evolution Strategies as a Scalable Alternative to Reinforcement Learning Tim Salimans Jonathan Ho Xi Chen Szymon Sidor Ilya Sutskever 72 1,523 0 10 Mar 2017
Evolving Deep Neural Networks Risto Miikkulainen J. Liang Elliot Meyerson Aditya Rawal Daniel Fink ... B. Raju Hormoz Shahrzad Arshak Navruzyan Nigel P. Duffy Babak Hodjat 74 886 0 01 Mar 2017
The Power of Sparsity in Convolutional Neural Networks Soravit Changpinyo Mark Sandler A. Zhmoginov 48 132 0 21 Feb 2017
Neural networks with differentiable structure Thomas Miconi 55 13 0 20 Jun 2016
Theano: A Python framework for fast computation of mathematical expressions The Theano Development Team Rami Al-Rfou Guillaume Alain Amjad Almahairi Christof Angermüller ... Kelvin Xu Lijun Xue Li Yao Saizheng Zhang Ying Zhang 135 2,338 0 09 May 2016
A topological insight into restricted Boltzmann machines Decebal Constantin Mocanu Elena Mocanu Phuong H. Nguyen M. Gibescu A. Liotta BDL 29 100 0 20 Apr 2016
Deep Learning with S-shaped Rectified Linear Activation Units Xiaojie Jin Chunyan Xu Jiashi Feng Yunchao Wei Junjun Xiong Shuicheng Yan 194 217 0 22 Dec 2015
How far can we go without convolution: Improving fully-connected networks Zhouhan Lin Roland Memisevic K. Konda 54 50 0 09 Nov 2015
Learning both Weights and Connections for Efficient Neural Networks Song Han Jeff Pool J. Tran W. Dally CVBM 247 6,628 0 08 Jun 2015
Fast ConvNets Using Group-wise Brain Damage V. Lebedev Victor Lempitsky AAML 119 448 0 08 Jun 2015