Stochasticity helps to navigate rough landscapes: comparing
gradient-descent-based algorithms in the phase retrieval problem

v1v2 (latest)

Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem

8 March 2021

Francesca Mignacco

Pierfrancesco Urbani

Lenka Zdeborová

ArXiv (abs)PDF HTML

Papers citing "Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem"

11 / 11 papers shown

Title
Bilinear Sequence Regression: A Model for Learning from Long Sequences of High-dimensional Tokens Vittorio Erba Emanuele Troiani Luca Biggio Antoine Maillard Lenka Zdeborová 202 2 0 24 Oct 2024
Stochastic Gradient Descent-like relaxation is equivalent to Metropolis dynamics in discrete optimization and inference problems Maria Chiara Angelini A. Cavaliere Raffaele Marino F. Ricci-Tersenghi 122 5 0 11 Sep 2023
Gradient flow in the gaussian covariate model: exact solution of learning curves and multiple descent structures Antione Bodin N. Macris 74 4 0 13 Dec 2022
Disordered Systems Insights on Computational Hardness D. Gamarnik Cristopher Moore Lenka Zdeborová AI4CE 84 36 0 15 Oct 2022
Rigorous dynamical mean field theory for stochastic gradient descent methods Cédric Gerbelot Emanuele Troiani Francesca Mignacco Florent Krzakala Lenka Zdeborova 114 29 0 12 Oct 2022
Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks Blake Bordelon Cengiz Pehlevan MLT 87 85 0 19 May 2022
The effective noise of Stochastic Gradient Descent Francesca Mignacco Pierfrancesco Urbani 69 39 0 20 Dec 2021
Model, sample, and epoch-wise descents: exact solution of gradient flow in the random feature model A. Bodin N. Macris 123 13 0 22 Oct 2021
The Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusion D. Kunin Javier Sagastuy-Breña Lauren Gillespie Eshed Margalit Hidenori Tanaka Surya Ganguli Daniel L. K. Yamins 93 20 0 19 Jul 2021
On the Cryptographic Hardness of Learning Single Periodic Neurons M. Song Ilias Zadik Joan Bruna AAML 68 28 0 20 Jun 2021
Analytical Study of Momentum-Based Acceleration Methods in Paradigmatic High-Dimensional Non-Convex Problems Stefano Sarao Mannelli Pierfrancesco Urbani 61 10 0 23 Feb 2021