On the Convergence of Adam under Non-uniform Smoothness: Separability from SGDM and Beyond

22 March 2024

Papers citing "On the Convergence of Adam under Non-uniform Smoothness: Separability from SGDM and Beyond"

5 / 5 papers shown

Title
Pushing the Limits of Low-Bit Optimizers: A Focus on EMA Dynamics Cong Xu Wenbin Liang Mo Yu Anan Liu K. Zhang Lizhuang Ma J. Wang J. Wang W. Zhang MQ 51 0 0 01 May 2025
Convergence Rate Analysis of LION Yiming Dong Huan Li Zhouchen Lin 30 0 0 12 Nov 2024
Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance Qi Zhang Yi Zhou Shaofeng Zou 27 3 0 01 Apr 2024
Convergence of Adam Under Relaxed Assumptions Haochuan Li Alexander Rakhlin Ali Jadbabaie 29 53 0 27 Apr 2023
A Simple Convergence Proof of Adam and Adagrad Alexandre Défossez Léon Bottou Francis R. Bach Nicolas Usunier 56 143 0 05 Mar 2020