SGD with a Constant Large Learning Rate Can Converge to Local Maxima

25 July 2021

Papers citing "SGD with a Constant Large Learning Rate Can Converge to Local Maxima"

6 / 6 papers shown

Title
Loss Landscape of Shallow ReLU-like Neural Networks: Stationary Points, Saddle Escape, and Network Embedding Zhengqing Wu Berfin Simsek Francois Ged ODL 38 0 0 08 Feb 2024
From Stability to Chaos: Analyzing Gradient Descent Dynamics in Quadratic Regression Xuxing Chen Krishnakumar Balasubramanian Promit Ghosal Bhavya Agrawalla 28 7 0 02 Oct 2023
What shapes the loss landscape of self-supervised learning? Liu Ziyin Ekdeep Singh Lubana Masakuni Ueda Hidenori Tanaka 48 20 0 02 Oct 2022
A sharp uniform-in-time error estimate for Stochastic Gradient Langevin Dynamics Lei Li Yuliang Wang 39 11 0 19 Jul 2022
The large learning rate phase of deep learning: the catapult mechanism Aitor Lewkowycz Yasaman Bahri Ethan Dyer Jascha Narain Sohl-Dickstein Guy Gur-Ari ODL 159 234 0 04 Mar 2020
Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition Hamed Karimi J. Nutini Mark W. Schmidt 133 1,198 0 16 Aug 2016