Stochastic optimization with momentum: convergence, fluctuations, and traps avoidance

7 December 2020

Papers citing "Stochastic optimization with momentum: convergence, fluctuations, and traps avoidance"

10 / 10 papers shown

Title
Sharp higher order convergence rates for the Adam optimizer Steffen Dereich Arnulf Jentzen Adrian Riekert ODL 61 0 0 28 Apr 2025
High-dimensional scaling limits and fluctuations of online least-squares SGD with smooth covariance Krishnakumar Balasubramanian Promit Ghosal Ye He 30 5 0 03 Apr 2023
Statistical Inference for Linear Functionals of Online SGD in High-dimensional Linear Regression Bhavya Agrawalla Krishnakumar Balasubramanian Promit Ghosal 23 2 0 20 Feb 2023
On the Algorithmic Stability and Generalization of Adaptive Optimization Methods Han Nguyen Hai Pham Sashank J. Reddi Barnabás Póczos ODL AI4CE 17 2 0 08 Nov 2022
Efficiency Ordering of Stochastic Gradient Descent Jie Hu Vishwaraj Doshi Do Young Eun 31 6 0 15 Sep 2022
Stable Anderson Acceleration for Deep Learning Massimiliano Lupo Pasini Junqi Yin Viktor Reshniak M. Stoyanov 15 4 0 26 Oct 2021
Stochastic Subgradient Descent on a Generic Definable Function Converges to a Minimizer S. Schechtman 22 1 0 06 Sep 2021
A new regret analysis for Adam-type algorithms Ahmet Alacaoglu Yura Malitsky P. Mertikopoulos V. Cevher ODL 48 42 0 21 Mar 2020
A Simple Convergence Proof of Adam and Adagrad Alexandre Défossez Léon Bottou Francis R. Bach Nicolas Usunier 56 143 0 05 Mar 2020
A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights Weijie Su Stephen P. Boyd Emmanuel J. Candes 105 1,152 0 04 Mar 2015