Smoothing the Landscape Boosts the Signal for SGD: Optimal Sample Complexity for Learning Single Index Models

18 May 2023

Papers citing "Smoothing the Landscape Boosts the Signal for SGD: Optimal Sample Complexity for Learning Single Index Models"

33 / 33 papers shown

Title
Survey on Algorithms for multi-index models Joan Bruna Daniel Hsu 18 0 0 07 Apr 2025
Feature learning from non-Gaussian inputs: the case of Independent Component Analysis in high dimensions Fabiola Ricci Lorenzo Bardone Sebastian Goldt OOD 28 0 0 31 Mar 2025
Learning a Single Index Model from Anisotropic Data with vanilla Stochastic Gradient Descent Guillaume Braun Minh Ha Quang Masaaki Imaizumi MLT 32 0 0 31 Mar 2025
Provable Benefits of Unsupervised Pre-training and Transfer Learning via Single-Index Models Taj Jones-McCormick Aukosh Jagannath S. Sen 36 0 0 24 Feb 2025
A distributional simplicity bias in the learning dynamics of transformers Riccardo Rende Federica Gerace A. Laio Sebastian Goldt 65 7 0 17 Feb 2025
Low-dimensional Functions are Efficiently Learnable under Randomly Biased Distributions Elisabetta Cornacchia Dan Mikulincer Elchanan Mossel 49 0 0 10 Feb 2025
Spectral Estimators for Multi-Index Models: Precise Asymptotics and Optimal Weak Recovery Filip Kovačević Yihan Zhang Marco Mondelli 65 0 0 03 Feb 2025
Gradient dynamics for low-rank fine-tuning beyond kernels Arif Kerem Dayi Sitan Chen 67 1 0 23 Nov 2024
Learning Gaussian Multi-Index Models with Gradient Flow: Time Complexity and Directional Convergence Berfin Simsek Amire Bendjeddou Daniel Hsu 32 0 0 13 Nov 2024
Sample and Computationally Efficient Robust Learning of Gaussian Single-Index Models Puqian Wang Nikos Zarifis Ilias Diakonikolas Jelena Diakonikolas 32 1 0 08 Nov 2024
Pretrained transformer efficiently learns low-dimensional target functions in-context Kazusato Oko Yujin Song Taiji Suzuki Denny Wu 23 4 0 04 Nov 2024
A Random Matrix Theory Perspective on the Spectrum of Learned Features and Asymptotic Generalization Capabilities Yatin Dandi Luca Pesce Hugo Cui Florent Krzakala Yue M. Lu Bruno Loureiro MLT 30 1 0 24 Oct 2024
Robust Feature Learning for Multi-Index Models in High Dimensions Alireza Mousavi-Hosseini Adel Javanmard Murat A. Erdogdu OOD AAML 37 1 0 21 Oct 2024
Learning Multi-Index Models with Neural Networks via Mean-Field Langevin Dynamics Alireza Mousavi-Hosseini Denny Wu Murat A. Erdogdu MLT AI4CE 27 6 0 14 Aug 2024
On the Complexity of Learning Sparse Functions with Statistical and Gradient Queries Nirmit Joshi Theodor Misiakiewicz Nathan Srebro 16 6 0 08 Jul 2024
Crafting Heavy-Tails in Weight Matrix Spectrum without Gradient Noise Vignesh Kothapalli Tianyu Pang Shenyang Deng Zongmin Liu Yaoqing Yang 21 3 0 07 Jun 2024
Online Learning and Information Exponents: On The Importance of Batch size, and Time/Complexity Tradeoffs Luca Arnaboldi Yatin Dandi Florent Krzakala Bruno Loureiro Luca Pesce Ludovic Stephan 35 1 0 04 Jun 2024
Neural network learns low-dimensional polynomials with SGD near the information-theoretic limit Jason D. Lee Kazusato Oko Taiji Suzuki Denny Wu MLT 71 20 0 03 Jun 2024
The High Line: Exact Risk and Learning Rate Curves of Stochastic Adaptive Learning Rate Algorithms Elizabeth Collins-Woodfin Inbar Seroussi Begona García Malaxechebarría Andrew W. Mackenzie Elliot Paquette Courtney Paquette 18 0 0 30 May 2024
Repetita Iuvant: Data Repetition Allows SGD to Learn High-Dimensional Multi-Index Functions Luca Arnaboldi Yatin Dandi Florent Krzakala Luca Pesce Ludovic Stephan 53 11 0 24 May 2024
Agnostic Active Learning of Single Index Models with Linear Sample Complexity Aarshvi Gajjar Wai Ming Tai Xingyu Xu Chinmay Hegde Yi Li Chris Musco 24 2 0 15 May 2024
Sliding down the stairs: how correlated latent variables accelerate learning with neural networks Lorenzo Bardone Sebastian Goldt 25 7 0 12 Apr 2024
The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents Yatin Dandi Emanuele Troiani Luca Arnaboldi Luca Pesce Lenka Zdeborová Florent Krzakala MLT 51 24 0 05 Feb 2024
Efficient Estimation of the Central Mean Subspace via Smoothed Gradient Outer Products Gan Yuan Mingyue Xu Samory Kpotufe Daniel Hsu 11 9 0 24 Dec 2023
Should Under-parameterized Student Networks Copy or Average Teacher Weights? Berfin Simsek Amire Bendjeddou W. Gerstner Johanni Brea 14 6 0 03 Nov 2023
Grokking as the Transition from Lazy to Rich Training Dynamics Tanishq Kumar Blake Bordelon Samuel Gershman C. Pehlevan 15 26 0 09 Oct 2023
Symmetric Single Index Learning Aaron Zweig Joan Bruna MLT 10 2 0 03 Oct 2023
Gradient-Based Feature Learning under Structured Data Alireza Mousavi-Hosseini Denny Wu Taiji Suzuki Murat A. Erdogdu MLT 10 18 0 07 Sep 2023
Pareto Frontiers in Neural Feature Learning: Data, Compute, Width, and Luck Benjamin L. Edelman Surbhi Goel Sham Kakade Eran Malach Cyril Zhang 32 7 0 07 Sep 2023
How Two-Layer Neural Networks Learn, One (Giant) Step at a Time Yatin Dandi Florent Krzakala Bruno Loureiro Luca Pesce Ludovic Stephan MLT 19 25 0 29 May 2023
SGD learning on neural networks: leap complexity and saddle-to-saddle dynamics Emmanuel Abbe Enric Boix-Adserà Theodor Misiakiewicz FedML MLT 76 72 0 21 Feb 2023
Learning Single-Index Models with Shallow Neural Networks A. Bietti Joan Bruna Clayton Sanford M. Song 147 65 0 27 Oct 2022
What Happens after SGD Reaches Zero Loss? --A Mathematical Framework Zhiyuan Li Tianhao Wang Sanjeev Arora MLT 83 98 0 13 Oct 2021