On a continuous time model of gradient descent dynamics and instability
in deep learning

On a continuous time model of gradient descent dynamics and instability in deep learning

3 February 2023

Papers citing "On a continuous time model of gradient descent dynamics and instability in deep learning"

11 / 11 papers shown

Title
Universal Sharpness Dynamics in Neural Network Training: Fixed Point Analysis, Edge of Stability, and Route to Chaos Dayal Singh Kalra Tianyu He M. Barkeshli 47 4 0 17 Feb 2025
Corridor Geometry in Gradient-Based Optimization Benoit Dherin M. Rosca 25 0 0 13 Feb 2024
Implicit biases in multitask and continual learning from a backward error analysis perspective Benoit Dherin 18 3 0 01 Nov 2023
Implicit regularisation in stochastic gradient descent: from single-objective to two-player games Mihaela Rosca M. Deisenroth 15 2 0 11 Jul 2023
Understanding Gradient Descent on Edge of Stability in Deep Learning Sanjeev Arora Zhiyuan Li A. Panigrahi MLT 72 88 0 19 May 2022
Stochastic Training is Not Necessary for Generalization Jonas Geiping Micah Goldblum Phillip E. Pope Michael Moeller Tom Goldstein 81 72 0 29 Sep 2021
Continuous vs. Discrete Optimization of Deep Neural Networks Omer Elkabetz Nadav Cohen 55 44 0 14 Jul 2021
The large learning rate phase of deep learning: the catapult mechanism Aitor Lewkowycz Yasaman Bahri Ethan Dyer Jascha Narain Sohl-Dickstein Guy Gur-Ari ODL 150 232 0 04 Mar 2020
Bag of Tricks for Image Classification with Convolutional Neural Networks Tong He Zhi-Li Zhang Hang Zhang Zhongyue Zhang Junyuan Xie Mu Li 210 1,391 0 04 Dec 2018
Representing smooth functions as compositions of near-identity functions with implications for deep network optimization Peter L. Bartlett S. Evans Philip M. Long 64 31 0 13 Apr 2018
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 273 2,878 0 15 Sep 2016