Quadratic models for understanding catapult dynamics of neural networks

24 May 2022

Papers citing "Quadratic models for understanding catapult dynamics of neural networks"

6 / 6 papers shown

Title
Universal Sharpness Dynamics in Neural Network Training: Fixed Point Analysis, Edge of Stability, and Route to Chaos Dayal Singh Kalra Tianyu He M. Barkeshli 49 4 0 17 Feb 2025
High dimensional analysis reveals conservative sharpening and a stochastic edge of stability Atish Agarwala Jeffrey Pennington 41 3 0 30 Apr 2024
Catapult Dynamics and Phase Transitions in Quadratic Nets David Meltzer Junyu Liu 25 9 0 18 Jan 2023
Second-order regression models exhibit progressive sharpening to the edge of stability Atish Agarwala Fabian Pedregosa Jeffrey Pennington 25 26 0 10 Oct 2022
Increasing Depth Leads to U-Shaped Test Risk in Over-parameterized Convolutional Networks Eshaan Nichani Adityanarayanan Radhakrishnan Caroline Uhler 16 9 0 19 Oct 2020
The large learning rate phase of deep learning: the catapult mechanism Aitor Lewkowycz Yasaman Bahri Ethan Dyer Jascha Narain Sohl-Dickstein Guy Gur-Ari ODL 159 234 0 04 Mar 2020