Gradient Descent with Polyak's Momentum Finds Flatter Minima via Large Catapults

25 November 2023

Papers citing "Gradient Descent with Polyak's Momentum Finds Flatter Minima via Large Catapults"

4 / 4 papers shown

Title
Understanding Edge-of-Stability Training Dynamics with a Minimalist Example Xingyu Zhu Zixuan Wang Xiang Wang Mo Zhou Rong Ge 59 35 0 07 Oct 2022
Understanding Gradient Descent on Edge of Stability in Deep Learning Sanjeev Arora Zhiyuan Li A. Panigrahi MLT 72 88 0 19 May 2022
Does Momentum Change the Implicit Regularization on Separable Data? Bohan Wang Qi Meng Huishuai Zhang Ruoyu Sun Wei-Neng Chen Zhirui Ma Tie-Yan Liu 39 15 0 08 Oct 2021
The large learning rate phase of deep learning: the catapult mechanism Aitor Lewkowycz Yasaman Bahri Ethan Dyer Jascha Narain Sohl-Dickstein Guy Gur-Ari ODL 150 198 0 04 Mar 2020