v1v2v3v4v5 (latest)

Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizers

12 October 2019

B. Bah

Holger Rauhut

Ulrich Terstiege

Michael Westdickenberg

MLT

ArXiv (abs)PDF HTML

Papers citing "Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizers"

46 / 46 papers shown

Title
Understanding Learning Invariance in Deep Linear Networks Hao Duan Guido Montúfar 25 0 0 16 Jun 2025
Symmetry in Neural Network Parameter Spaces Bo Zhao Robin Walters Rose Yu 27 0 0 16 Jun 2025
Transformative or Conservative? Conservation laws for ResNets and Transformers Sibylle Marcotte Rémi Gribonval Gabriel Peyré 40 0 0 06 Jun 2025
Digital implementations of deep feature extractors are intrinsically informative Max Getter 130 0 0 20 Feb 2025
A new Input Convex Neural Network with application to options pricing Vincent Lemaire Gilles Pagès Christian Yeo 99 0 0 19 Nov 2024
Plastic Learning with Deep Fourier Features Alex Lewandowski Dale Schuurmans Marlos C. Machado CLL 102 3 0 27 Oct 2024
Convergence of continuous-time stochastic gradient descent with applications to linear deep neural networks Gabor Lugosi Eulalia Nualart 35 0 0 11 Sep 2024
Lecture Notes on Linear Neural Networks: A Tale of Optimization and Generalization in Deep Learning Nadav Cohen Noam Razin 115 0 0 25 Aug 2024
Federated Dynamical Low-Rank Training with Global Loss Convergence Guarantees Steffen Schotthöfer M. P. Laiu FedML 75 5 0 25 Jun 2024
Keep the Momentum: Conservation Laws beyond Euclidean Gradient Flows Sibylle Marcotte Rémi Gribonval Gabriel Peyré 48 1 0 21 May 2024
Understanding the training of infinitely deep and wide ResNets with Conditional Optimal Transport Raphael Barboni Gabriel Peyré Franccois-Xavier Vialard 63 3 0 19 Mar 2024
On the Stability of Gradient Descent for Large Learning Rate Alexandru Cruaciun Debarghya Ghoshdastidar MLT ODL 27 0 0 20 Feb 2024
Neural Rank Collapse: Weight Decay and Small Within-Class Variability Yield Low-Rank Bias Emanuele Zangrando Piero Deidda Simone Brugiapaglia Nicola Guglielmi Francesco Tudisco 83 8 0 06 Feb 2024
On the Role of Initialization on the Implicit Bias in Deep Linear Networks Oria Gruber H. Avron AI4CE 16 0 0 04 Feb 2024
On the Impact of Overparameterization on the Training of a Shallow Neural Network in High Dimensions Simon Martin Francis Bach Giulio Biroli 104 11 0 07 Nov 2023
Implicit regularization in AI meets generalized hardness of approximation in optimization -- Sharp results for diagonal linear networks J. S. Wind Vegard Antun A. Hansen 64 4 0 13 Jul 2023
Neural Hilbert Ladders: Multi-Layer Neural Networks in Function Space Zhengdao Chen 90 1 0 03 Jul 2023
Abide by the Law and Follow the Flow: Conservation Laws for Gradient Flows Sibylle Marcotte Rémi Gribonval Gabriel Peyré 117 19 0 30 Jun 2023
Robust low-rank training via approximate orthonormal constraints Dayana Savostianova Emanuele Zangrando Gianluca Ceruti Francesco Tudisco 69 10 0 02 Jun 2023
Combining Explicit and Implicit Regularization for Efficient Learning in Deep Networks Dan Zhao 111 6 0 01 Jun 2023
Neural (Tangent Kernel) Collapse Mariia Seleznova Dana Weitzner Raja Giryes Gitta Kutyniok H. Chou 64 10 0 25 May 2023
Robust Implicit Regularization via Weight Normalization H. Chou Holger Rauhut Rachel A. Ward 88 7 0 09 May 2023
Function Space and Critical Points of Linear Convolutional Networks Kathlén Kohn Guido Montúfar Vahid Shahverdi Matthew Trager 85 14 0 12 Apr 2023
Optimization Dynamics of Equivariant and Augmented Neural Networks Axel Flinth F. Ohlsson 105 7 0 23 Mar 2023
Critical Points and Convergence Analysis of Generative Deep Linear Networks Trained with Bures-Wasserstein Loss Pierre Bréchet Katerina Papagiannouli Jing An Guido Montúfar 82 4 0 06 Mar 2023
A Dynamics Theory of Implicit Regularization in Deep Low-Rank Matrix Factorization JIAN-PENG Cao Chao Qian Yihui Huang Dicheng Chen Yuncheng Gao Jiyang Dong D. Guo X. Qu 122 1 0 29 Dec 2022
Infinite-width limit of deep linear neural networks Lénaïc Chizat Maria Colombo Xavier Fernández-Real Alessio Figalli 84 16 0 29 Nov 2022
Finite Sample Identification of Wide Shallow Neural Networks with Biases M. Fornasier T. Klock Marco Mondelli Michael Rauchensteiner 52 6 0 08 Nov 2022
Deep Linear Networks for Matrix Completion -- An Infinite Depth Limit Nadav Cohen Govind Menon Zsolt Veraszto ODL 51 7 0 22 Oct 2022
Gradient descent provably escapes saddle points in the training of shallow ReLU networks Patrick Cheridito Arnulf Jentzen Florian Rossmannek 103 5 0 03 Aug 2022
Side Effects of Learning from Low-dimensional Data Embedded in a Euclidean Space Juncai He R. Tsai Rachel A. Ward 95 9 0 01 Mar 2022
Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks Noam Razin Asaf Maman Nadav Cohen 132 29 0 27 Jan 2022
Global Convergence Analysis of Deep Linear Networks with A One-neuron Layer Kun Chen Dachao Lin Zhihua Zhang 60 1 0 08 Jan 2022
How and When Random Feedback Works: A Case Study of Low-Rank Matrix Factorization Shivam Garg Santosh Vempala 78 3 0 17 Nov 2021
Convergence of gradient descent for learning linear neural networks Gabin Maxime Nguegnang Holger Rauhut Ulrich Terstiege MLT 63 18 0 04 Aug 2021
Geometry of Linear Convolutional Networks Kathlén Kohn Thomas Merkh Guido Montúfar Matthew Trager 115 20 0 03 Aug 2021
The loss landscape of deep linear neural networks: a second-order analysis El Mehdi Achour Franccois Malgouyres Sébastien Gerchinovitz ODL 69 10 0 28 Jul 2021
Convergence analysis for gradient flows in the training of artificial neural networks with ReLU activation Arnulf Jentzen Adrian Riekert 55 23 0 09 Jul 2021
Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction Dominik Stöger Mahdi Soltanolkotabi ODL 96 78 0 28 Jun 2021
Principal Components Bias in Over-parameterized Linear Models, and its Manifestation in Deep Neural Networks Guy Hacohen D. Weinshall 125 10 0 12 May 2021
Stable Recovery of Entangled Weights: Towards Robust Identification of Deep Neural Networks from Minimal Samples Christian Fiedler M. Fornasier T. Klock Michael Rauchensteiner OOD 45 13 0 18 Jan 2021
First-Order Methods for Convex Optimization Pavel Dvurechensky Mathias Staudigl Shimrit Shtern ODL 90 26 0 04 Jan 2021
Asymptotic convergence rate of Dropout on shallow linear neural networks Albert Senen-Cerda J. Sanders 118 8 0 01 Dec 2020
Gradient Descent for Deep Matrix Factorization: Dynamics and Implicit Bias towards Low Rank H. Chou Carsten Gieshoff J. Maly Holger Rauhut 83 42 0 27 Nov 2020
Consensus-Based Optimization on the Sphere: Convergence to Global Minimizers and Machine Learning M. Fornasier Hui Huang L. Pareschi Philippe Sünnen 129 69 0 31 Jan 2020
Pure and Spurious Critical Points: a Geometric Study of Linear Networks Matthew Trager Kathlén Kohn Joan Bruna 77 31 0 03 Oct 2019