Convergence of stochastic gradient descent schemes for Lojasiewicz-landscapes

16 February 2021

Papers citing "Convergence of stochastic gradient descent schemes for Lojasiewicz-landscapes"

21 / 21 papers shown

Title
Sharp higher order convergence rates for the Adam optimizer Steffen Dereich Arnulf Jentzen Adrian Riekert ODL 61 0 0 28 Apr 2025
In almost all shallow analytic neural network optimization landscapes, efficient minimizers have strongly convex neighborhoods Felix Benning Steffen Dereich ODL 31 0 0 11 Apr 2025
Momentum-based minimization of the Ginzburg-Landau functional on Euclidean spaces and graphs Oluwatosin Akande Patrick Dondl Kanan Gupta Akwum Onwunta Stephan Wojtowytsch 23 1 0 03 Jan 2025
Polyak's Heavy Ball Method Achieves Accelerated Local Rate of Convergence under Polyak-Lojasiewicz Inequality Sebastian Kassing Simon Weissmann 21 1 0 22 Oct 2024
Mask in the Mirror: Implicit Sparsification Tom Jacobs R. Burkholz 40 3 0 19 Aug 2024
Almost sure convergence rates of stochastic gradient methods under gradient domination Simon Weissmann Sara Klein Waïss Azizian Leif Döring 28 3 0 22 May 2024
Singular-limit analysis of gradient descent with noise injection Anna Shalova André Schlichting M. Peletier 35 1 0 18 Apr 2024
A New Random Reshuffling Method for Nonsmooth Nonconvex Finite-sum Optimization Junwen Qiu Xiao Li Andre Milzarek 37 2 0 02 Dec 2023
On the existence of optimal shallow feedforward networks with ReLU activation Steffen Dereich Sebastian Kassing 14 4 0 06 Mar 2023
On the existence of minimizers in shallow residual ReLU neural network optimization landscapes Steffen Dereich Arnulf Jentzen Sebastian Kassing 15 6 0 28 Feb 2023
$On bounds for norms of reparameterized ReLU artificial neural network parameters: sums of fractional powers of the Lipschitz norm control the network parameter vector$ On bounds for norms of reparameterized ReLU artificial neural network parameters: sums of fractional powers of the Lipschitz norm control the network parameter vector Arnulf Jentzen T. Kröger 22 0 0 27 Jun 2022
A Local Convergence Theory for the Stochastic Gradient Descent Method in Non-Convex Optimization With Non-isolated Local Minima Tae-Eon Ko Xiantao Li 20 2 0 21 Mar 2022
Convergence proof for stochastic gradient descent in the training of deep neural networks with ReLU activation for constant target functions Martin Hutzenthaler Arnulf Jentzen Katharina Pohl Adrian Riekert Luca Scarpa MLT 32 6 0 13 Dec 2021
Existence, uniqueness, and convergence rates for gradient flows in the training of artificial neural networks with ReLU activation Simon Eberle Arnulf Jentzen Adrian Riekert G. Weiss 26 12 0 18 Aug 2021
A proof of convergence for the gradient descent optimization method with random initializations in the training of neural networks with ReLU activation for piecewise linear target functions Arnulf Jentzen Adrian Riekert 23 13 0 10 Aug 2021
Stochastic gradient descent with noise of machine learning type. Part I: Discrete time analysis Stephan Wojtowytsch 21 50 0 04 May 2021
A proof of convergence for stochastic gradient descent in the training of artificial neural networks with ReLU activation for constant target functions Arnulf Jentzen Adrian Riekert MLT 21 13 0 01 Apr 2021
A Retrospective Approximation Approach for Smooth Stochastic Optimization David Newton Raghu Bollapragada R. Pasupathy N. Yip 22 2 0 07 Mar 2021
Convergence rates for gradient descent in the training of overparameterized artificial neural networks with biases Arnulf Jentzen T. Kröger ODL 20 7 0 23 Feb 2021
Newton-MR: Inexact Newton Method With Minimum Residual Sub-problem Solver Fred Roosta Yang Liu Peng Xu Michael W. Mahoney 8 12 0 30 Sep 2018
Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition Hamed Karimi J. Nutini Mark W. Schmidt 119 1,198 0 16 Aug 2016