v1v2v3v4 (latest)

Passed & Spurious: Descent Algorithms and Local Minima in Spiked Matrix-Tensor Models

1 February 2019

Stefano Sarao Mannelli

Florent Krzakala

Pierfrancesco Urbani

Lenka Zdeborová

ArXiv (abs)PDF HTML

Papers citing "Passed & Spurious: Descent Algorithms and Local Minima in Spiked Matrix-Tensor Models"

31 / 31 papers shown

Title
Computational Thresholds in Multi-Modal Learning via the Spiked Matrix-Tensor Model Hugo Tabanelli Pierre Mergny Lenka Zdeborová Florent Krzakala 50 0 0 03 Jun 2025
Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer Blake Bordelon Cengiz Pehlevan AI4CE 241 1 0 04 Feb 2025
How Feature Learning Can Improve Neural Scaling Laws Blake Bordelon Alexander B. Atanasov Cengiz Pehlevan 144 17 0 26 Sep 2024
Infinite Limits of Multi-head Transformer Dynamics Blake Bordelon Hamza Tahir Chaudhry Cengiz Pehlevan AI4CE 117 14 0 24 May 2024
Sliding down the stairs: how correlated latent variables accelerate learning with neural networks Lorenzo Bardone Sebastian Goldt 78 7 0 12 Apr 2024
From Zero to Hero: How local curvature at artless initial conditions leads away from bad minima Tony Bonnaire Giulio Biroli C. Cammarota 113 0 0 04 Mar 2024
The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents Yatin Dandi Emanuele Troiani Luca Arnaboldi Luca Pesce Lenka Zdeborová Florent Krzakala MLT 120 30 0 05 Feb 2024
A Dynamical Model of Neural Scaling Laws Blake Bordelon Alexander B. Atanasov Cengiz Pehlevan 148 44 0 02 Feb 2024
Topological complexity of spiked random polynomials and finite-rank spherical integrals Vanessa Piccolo 47 1 0 19 Dec 2023
Stochastic Gradient Descent outperforms Gradient Descent in recovering a high-dimensional signal in a glassy energy landscape Persia Jana Kamali Pierfrancesco Urbani 75 6 0 09 Sep 2023
Gradient flow on extensive-rank positive semi-definite matrix denoising A. Bodin N. Macris 86 4 0 16 Mar 2023
Average-Case Complexity of Tensor Decomposition for Low-Degree Polynomials Alexander S. Wein 113 11 0 10 Nov 2022
Learning Single-Index Models with Shallow Neural Networks A. Bietti Joan Bruna Clayton Sanford M. Song 225 71 0 27 Oct 2022
On free energy barriers in Gaussian priors and failure of cold start MCMC for high-dimensional unimodal distributions Afonso S. Bandeira Antoine Maillard Richard Nickl Sven Wang 83 10 0 05 Sep 2022
Sudakov-Fernique post-AMP, and a new proof of the local convexity of the TAP free energy Michael Celentano 94 21 0 19 Aug 2022
Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks Blake Bordelon Cengiz Pehlevan MLT 87 85 0 19 May 2022
High-dimensional Asymptotics of Langevin Dynamics in Spiked Matrix Models Tengyuan Liang Subhabrata Sen Pragya Sur 81 7 0 09 Apr 2022
Selective Multiple Power Iteration: from Tensor PCA to gradient-based exploration of landscapes M. Ouerfelli M. Tamaazousti V. Rivasseau 75 7 0 23 Dec 2021
Statistical limits of dictionary learning: random matrix theory and the spectral replica method Jean Barbier N. Macris 119 26 0 14 Sep 2021
An Analytical Theory of Curriculum Learning in Teacher-Student Networks Luca Saglietti Stefano Sarao Mannelli Andrew M. Saxe 57 26 0 15 Jun 2021
Rank-one matrix estimation: analytic time evolution of gradient descent dynamics A. Bodin N. Macris 49 5 0 25 May 2021
Analytical Study of Momentum-Based Acceleration Methods in Paradigmatic High-Dimensional Non-Convex Problems Stefano Sarao Mannelli Pierfrancesco Urbani 61 10 0 23 Feb 2021
A spin-glass model for the loss surfaces of generative adversarial networks Nicholas P. Baskerville J. Keating F. Mezzadri J. Najnudel GAN 88 12 0 07 Jan 2021
Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions Stefano Sarao Mannelli Eric Vanden-Eijnden Lenka Zdeborová AI4CE 79 49 0 27 Jun 2020
Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification Francesca Mignacco Florent Krzakala Pierfrancesco Urbani Lenka Zdeborová MLT 88 69 0 10 Jun 2020
The Loss Surfaces of Neural Networks with General Activation Functions Nicholas P. Baskerville J. Keating F. Mezzadri J. Najnudel ODL AI4CE 133 26 0 08 Apr 2020
Online stochastic gradient descent on non-convex losses from high-dimensional inference Gerard Ben Arous Reza Gheissari Aukosh Jagannath 108 91 0 23 Mar 2020
Iterative Averaging in the Quest for Best Test Error Diego Granziol Xingchen Wan Samuel Albanie Stephen J. Roberts 66 3 0 02 Mar 2020
Thresholds of descending algorithms in inference problems Stefano Sarao Mannelli Lenka Zdeborova AI4CE 62 4 0 02 Jan 2020
Notes on Computational Hardness of Hypothesis Testing: Predictions using the Low-Degree Likelihood Ratio Dmitriy Kunisky Alexander S. Wein Afonso S. Bandeira 99 148 0 26 Jul 2019
Who is Afraid of Big Bad Minima? Analysis of Gradient-Flow in a Spiked Matrix-Tensor Model Stefano Sarao Mannelli Giulio Biroli C. Cammarota Florent Krzakala Lenka Zdeborová 62 43 0 18 Jul 2019