Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow
in Phase Retrieval

Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval

12 June 2020

Stefano Sarao Mannelli

Florent Krzakala

Pierfrancesco Urbani

Lenka Zdeborová

ArXiv (abs)PDF HTML

Papers citing "Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval"

10 / 10 papers shown

Title
Bilinear Sequence Regression: A Model for Learning from Long Sequences of High-dimensional Tokens Vittorio Erba Emanuele Troiani Luca Biggio Antoine Maillard Lenka Zdeborová 202 2 0 24 Oct 2024
Sampling with flows, diffusion and autoregressive neural networks: A spin-glass perspective Davide Ghio Yatin Dandi Florent Krzakala Lenka Zdeborová DiffM 67 29 0 27 Aug 2023
Initial Guessing Bias: How Untrained Networks Favor Some Classes Emanuele Francazi Aurelien Lucchi Marco Baity-Jesi AI4CE 75 4 0 01 Jun 2023
Sudakov-Fernique post-AMP, and a new proof of the local convexity of the TAP free energy Michael Celentano 94 21 0 19 Aug 2022
On the Cryptographic Hardness of Learning Single Periodic Neurons M. Song Ilias Zadik Joan Bruna AAML 68 28 0 20 Jun 2021
An Analytical Theory of Curriculum Learning in Teacher-Student Networks Luca Saglietti Stefano Sarao Mannelli Andrew M. Saxe 55 26 0 15 Jun 2021
Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem Francesca Mignacco Pierfrancesco Urbani Lenka Zdeborová 85 36 0 08 Mar 2021
Analytical Study of Momentum-Based Acceleration Methods in Paradigmatic High-Dimensional Non-Convex Problems Stefano Sarao Mannelli Pierfrancesco Urbani 53 10 0 23 Feb 2021
Quickly Finding a Benign Region via Heavy Ball Momentum in Non-Convex Optimization Jun-Kun Wang Jacob D. Abernethy 108 7 0 04 Oct 2020
Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions Stefano Sarao Mannelli Eric Vanden-Eijnden Lenka Zdeborová AI4CE 73 49 0 27 Jun 2020