Gradient Dynamics of Shallow Univariate ReLU Networks

Gradient Dynamics of Shallow Univariate ReLU Networks

18 June 2019

Francis Williams

Matthew Trager

Claudio Silva

Daniele Panozzo

Joan Bruna

Papers citing "Gradient Dynamics of Shallow Univariate ReLU Networks"

19 / 19 papers shown

Title
Gradient Descent as a Shrinkage Operator for Spectral Bias Simon Lucey 38 0 0 25 Apr 2025
An Invitation to Neuroalgebraic Geometry G. Marchetti V. Shahverdi Stefano Mereta Matthew Trager Kathlén Kohn 119 2 0 31 Jan 2025
Equidistribution-based training of Free Knot Splines and ReLU Neural Networks Simone Appella S. Arridge Chris Budd Teo Deveney L. Kreusser 38 0 0 02 Jul 2024
Lazy vs hasty: linearization in deep networks impacts learning schedule based on example difficulty Thomas George Guillaume Lajoie A. Baratin 31 5 0 19 Sep 2022
On the Implicit Bias in Deep-Learning Algorithms Gal Vardi FedML AI4CE 34 72 0 26 Aug 2022
$Intrinsic dimensionality and generalization properties of the $\mathcal{R}$-norm inductive bias$ Intrinsic dimensionality and generalization properties of the $\mathcal{R}$ -norm inductive bias Navid Ardeshir Daniel J. Hsu Clayton Sanford CML AI4CE 18 6 0 10 Jun 2022
On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias Itay Safran Gal Vardi Jason D. Lee MLT 59 23 0 18 May 2022
Implicit Bias of MSE Gradient Optimization in Underparameterized Neural Networks Benjamin Bowman Guido Montúfar 28 11 0 12 Jan 2022
Neural Fields as Learnable Kernels for 3D Reconstruction Francis Williams Zan Gojcic S. Khamis Denis Zorin Joan Bruna Sanja Fidler Or Litany 50 66 0 26 Nov 2021
Mean-field Analysis of Piecewise Linear Solutions for Wide ReLU Networks A. Shevchenko Vyacheslav Kungurtsev Marco Mondelli MLT 41 13 0 03 Nov 2021
Convergence analysis for gradient flows in the training of artificial neural networks with ReLU activation Arnulf Jentzen Adrian Riekert 27 23 0 09 Jul 2021
On Energy-Based Models with Overparametrized Shallow Neural Networks Carles Domingo-Enrich A. Bietti Eric Vanden-Eijnden Joan Bruna BDL 33 9 0 15 Apr 2021
Neural Splines: Fitting 3D Surfaces with Infinitely-Wide Neural Networks Francis Williams Matthew Trager Joan Bruna Denis Zorin 21 67 0 24 Jun 2020
When Does Preconditioning Help or Hurt Generalization? S. Amari Jimmy Ba Roger C. Grosse Xuechen Li Atsushi Nitanda Taiji Suzuki Denny Wu Ji Xu 36 32 0 18 Jun 2020
Convex Geometry and Duality of Over-parameterized Neural Networks Tolga Ergen Mert Pilanci MLT 42 54 0 25 Feb 2020
Optimization for deep learning: theory and algorithms Ruoyu Sun ODL 19 168 0 19 Dec 2019
Neural Spectrum Alignment: Empirical Study Dmitry Kopitkov Vadim Indelman 29 14 0 19 Oct 2019
Kernel and Rich Regimes in Overparametrized Models Blake E. Woodworth Suriya Gunasekar Pedro H. P. Savarese E. Moroshko Itay Golan J. Lee Daniel Soudry Nathan Srebro 24 352 0 13 Jun 2019
Universal Approximation with Deep Narrow Networks Patrick Kidger Terry Lyons 34 327 0 21 May 2019