From Stability to Chaos: Analyzing Gradient Descent Dynamics in
Quadratic Regression

From Stability to Chaos: Analyzing Gradient Descent Dynamics in Quadratic Regression

2 October 2023

Krishnakumar Balasubramanian

Bhavya Agrawalla

Papers citing "From Stability to Chaos: Analyzing Gradient Descent Dynamics in Quadratic Regression"

10 / 10 papers shown

Title
Minimax Optimal Convergence of Gradient Descent in Logistic Regression via Large and Adaptive Stepsizes Ruiqi Zhang Jingfeng Wu Licong Lin Peter L. Bartlett 20 0 0 05 Apr 2025
Universal Sharpness Dynamics in Neural Network Training: Fixed Point Analysis, Edge of Stability, and Route to Chaos Dayal Singh Kalra Tianyu He M. Barkeshli 47 4 0 17 Feb 2025
$The boundary of neural network trainability is fractal$ The boundary of neural network trainability is fractal Jascha Narain Sohl-Dickstein 16 8 0 09 Feb 2024
Understanding Edge-of-Stability Training Dynamics with a Minimalist Example Xingyu Zhu Zixuan Wang Xiang Wang Mo Zhou Rong Ge 62 35 0 07 Oct 2022
Training Scale-Invariant Neural Networks on the Sphere Can Happen in Three Regimes M. Kodryan E. Lobacheva M. Nakhodnov Dmitry Vetrov 26 15 0 08 Sep 2022
Chaotic Regularization and Heavy-Tailed Limits for Deterministic Gradient Descent S. H. Lim Yijun Wan Umut cSimcsekli 24 12 0 23 May 2022
Understanding Gradient Descent on Edge of Stability in Deep Learning Sanjeev Arora Zhiyuan Li A. Panigrahi MLT 72 88 0 19 May 2022
Neural Network Weights Do Not Converge to Stationary Points: An Invariant Measure Perspective J. Zhang Haochuan Li S. Sra Ali Jadbabaie 66 9 0 12 Oct 2021
Large Learning Rate Tames Homogeneity: Convergence and Balancing Effect Yuqing Wang Minshuo Chen T. Zhao Molei Tao AI4CE 55 40 0 07 Oct 2021
The large learning rate phase of deep learning: the catapult mechanism Aitor Lewkowycz Yasaman Bahri Ethan Dyer Jascha Narain Sohl-Dickstein Guy Gur-Ari ODL 150 232 0 04 Mar 2020