SGD with Clipping is Secretly Estimating the Median Gradient

20 February 2024

Papers citing "SGD with Clipping is Secretly Estimating the Median Gradient"

2 / 2 papers shown

Title
Revisiting Gradient Clipping: Stochastic bias and tight convergence guarantees Anastasia Koloskova Hadrien Hendrikx Sebastian U. Stich 112 49 0 02 May 2023
Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be Frederik Kunstner Jacques Chen J. Lavington Mark W. Schmidt 40 67 0 27 Apr 2023