Escaping Saddle Points with Adaptive Gradient Methods

Escaping Saddle Points with Adaptive Gradient Methods

26 January 2019

Sashank J. Reddi

Sanjiv Kumar

Papers citing "Escaping Saddle Points with Adaptive Gradient Methods"

12 / 12 papers shown

Title
Particle Semi-Implicit Variational Inference Jen Ning Lim A. M. Johansen 59 4 0 30 Jun 2024
Regularized Gradient Clipping Provably Trains Wide and Deep Neural Networks Matteo Tucat Anirbit Mukherjee Procheta Sen Mingfei Sun Omar Rivasplata MLT 39 1 0 12 Apr 2024
The Expected Loss of Preconditioned Langevin Dynamics Reveals the Hessian Rank Amitay Bar Rotem Mulayoff T. Michaeli Ronen Talmon 66 0 0 21 Feb 2024
QLABGrad: a Hyperparameter-Free and Convergence-Guaranteed Scheme for Deep Learning Minghan Fu Fang-Xiang Wu ODL 42 7 0 01 Feb 2023
TiAda: A Time-scale Adaptive Algorithm for Nonconvex Minimax Optimization Xiang Li Junchi Yang Niao He 34 8 0 31 Oct 2022
Statistical inference of travelers' route choice preferences with system-level data Pablo Guarda Sean Qian 19 6 0 23 Apr 2022
Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization Zeke Xie Li-xin Yuan Zhanxing Zhu Masashi Sugiyama 32 29 0 31 Mar 2021
Quickly Finding a Benign Region via Heavy Ball Momentum in Non-Convex Optimization Jun-Kun Wang Jacob D. Abernethy 24 7 0 04 Oct 2020
Riemannian stochastic recursive momentum method for non-convex optimization Andi Han Junbin Gao ODL 28 17 0 11 Aug 2020
Adaptive Federated Optimization Sashank J. Reddi Zachary B. Charles Manzil Zaheer Zachary Garrett Keith Rush Jakub Konecný Sanjiv Kumar H. B. McMahan FedML 58 1,395 0 29 Feb 2020
Hessian based analysis of SGD for Deep Nets: Dynamics and Generalization Xinyan Li Qilong Gu Yingxue Zhou Tiancong Chen A. Banerjee ODL 42 51 0 24 Jul 2019
Why gradient clipping accelerates training: A theoretical justification for adaptivity J.N. Zhang Tianxing He S. Sra Ali Jadbabaie 30 446 0 28 May 2019