Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regions

2 May 2016

Papers citing "Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regions"

19 / 19 papers shown

Title
Langevin Multiplicative Weights Update with Applications in Polynomial Portfolio Management Yi-Hu Feng Xiao Wang Tian Xie 62 0 0 26 Feb 2025
Training-set-free two-stage deep learning for spectroscopic data de-noising Dongchen Huang Junde Liu Tian Qian Hongming Weng 36 0 0 29 Feb 2024
A PAC-Bayesian Link Between Generalisation and Flat Minima Maxime Haddouche Paul Viallard Umut Simsekli Benjamin Guedj 40 3 0 13 Feb 2024
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction Kaifeng Lyu Zhiyuan Li Sanjeev Arora FAtt 37 69 0 14 Jun 2022
Understanding the unstable convergence of gradient descent Kwangjun Ahn J. Zhang S. Sra 24 57 0 03 Apr 2022
Over-Parametrized Matrix Factorization in the Presence of Spurious Stationary Points Armin Eftekhari 19 1 0 25 Dec 2021
Convergence proof for stochastic gradient descent in the training of deep neural networks with ReLU activation for constant target functions Martin Hutzenthaler Arnulf Jentzen Katharina Pohl Adrian Riekert Luca Scarpa MLT 34 6 0 13 Dec 2021
New Q-Newton's method meets Backtracking line search: good convergence guarantee, saddle points avoidance, quadratic rate of convergence, and easy implementation T. Truong 14 5 0 23 Aug 2021
Global Convergence of Gradient Descent for Asymmetric Low-Rank Matrix Factorization Tian-Chun Ye S. Du 19 46 0 27 Jun 2021
A proof of convergence for stochastic gradient descent in the training of artificial neural networks with ReLU activation for constant target functions Arnulf Jentzen Adrian Riekert MLT 32 13 0 01 Apr 2021
Landscape analysis for shallow neural networks: complete classification of critical points for affine target functions Patrick Cheridito Arnulf Jentzen Florian Rossmannek 24 10 0 19 Mar 2021
On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems P. Mertikopoulos Nadav Hallak Ali Kavis V. Cevher 11 85 0 19 Jun 2020
On the Impossibility of Global Convergence in Multi-Loss Optimization Alistair Letcher 13 32 0 26 May 2020
A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks Sanjeev Arora Nadav Cohen Noah Golowich Wei Hu 11 280 0 04 Oct 2018
Diffusion Approximations for Online Principal Component Estimation and Global Convergence C. J. Li Mengdi Wang Han Liu Tong Zhang 26 12 0 29 Aug 2018
Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond Levent Sagun Léon Bottou Yann LeCun UQCV 16 227 0 22 Nov 2016
Local Maxima in the Likelihood of Gaussian Mixture Models: Structural Results and Algorithmic Consequences Chi Jin Yuchen Zhang Sivaraman Balakrishnan Martin J. Wainwright Michael I. Jordan 19 131 0 04 Sep 2016
Provable Efficient Online Matrix Completion via Non-convex Stochastic Gradient Descent Chi Jin Sham Kakade Praneeth Netrapalli 11 81 0 26 May 2016
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 179 1,185 0 30 Nov 2014