Accelerated Gradient Descent Escapes Saddle Points Faster than Gradient Descent

28 November 2017

Papers citing "Accelerated Gradient Descent Escapes Saddle Points Faster than Gradient Descent"

32 / 32 papers shown

Title
Langevin Multiplicative Weights Update with Applications in Polynomial Portfolio Management Yi-Hu Feng Xiao Wang Tian Xie 52 0 0 26 Feb 2025
Grams: Gradient Descent with Adaptive Momentum Scaling Yang Cao Xiaoyu Li Zhao-quan Song ODL 83 2 0 22 Dec 2024
Cautious Optimizers: Improving Training with One Line of Code Kaizhao Liang Lizhang Chen B. Liu Qiang Liu ODL 103 5 0 25 Nov 2024
Comparisons Are All You Need for Optimizing Smooth Functions Chenyi Zhang Tongyang Li AAML 24 1 0 19 May 2024
Beyond first-order methods for non-convex non-concave min-max optimization Abhijeet Vyas Brian Bullins 23 1 0 17 Apr 2023
Escaping From Saddle Points Using Asynchronous Coordinate Gradient Descent Marco Bornstein Jin-Peng Liu Jingling Li Furong Huang 13 0 0 17 Nov 2022
Zeroth-Order Negative Curvature Finding: Escaping Saddle Points without Gradients Hualin Zhang Huan Xiong Bin Gu 22 7 0 04 Oct 2022
On the fast convergence of minibatch heavy ball momentum Raghu Bollapragada Tyler Chen Rachel A. Ward 24 17 0 15 Jun 2022
An Adaptive Gradient Method with Energy and Momentum Hailiang Liu Xuping Tian ODL 16 9 0 23 Mar 2022
Tackling benign nonconvexity with smoothing and stochastic gradients Harsh Vardhan Sebastian U. Stich 16 8 0 18 Feb 2022
Differentially Private Temporal Difference Learning with Stochastic Nonconvex-Strongly-Concave Optimization Canzhe Zhao Yanjie Ze Jing Dong Baoxiang Wang Shuai Li 39 4 0 25 Jan 2022
Adaptive Gaussian Process based Stochastic Trajectory Optimization for Motion Planning Yichang Feng Haiyun Zhang Jin Wang Guodong Lu 33 30 0 30 Dec 2021
Faster Perturbed Stochastic Gradient Methods for Finding Local Minima Zixiang Chen Dongruo Zhou Quanquan Gu 25 1 0 25 Oct 2021
On the Convergence of Projected Alternating Maximization for Equitable and Optimal Transport Minhui Huang Shiqian Ma Lifeng Lai 27 3 0 29 Sep 2021
Majorization Minimization Methods for Distributed Pose Graph Optimization Taosha Fan Todd D. Murphey 31 19 0 30 Jul 2021
The loss landscape of deep linear neural networks: a second-order analysis E. M. Achour Franccois Malgouyres Sébastien Gerchinovitz ODL 22 9 0 28 Jul 2021
AEGD: Adaptive Gradient Descent with Energy Hailiang Liu Xuping Tian ODL 25 11 0 10 Oct 2020
Learning explanations that are hard to vary Giambattista Parascandolo Alexander Neitz Antonio Orvieto Luigi Gresele Bernhard Schölkopf FAtt 11 178 0 01 Sep 2020
Global Convergence of Policy Gradient Methods to (Almost) Locally Optimal Policies K. Zhang Alec Koppel Haoqi Zhu Tamer Basar 14 186 0 19 Jun 2019
Stabilized SVRG: Simple Variance Reduction for Nonconvex Optimization Rong Ge Zhize Li Weiyao Wang Xiang Wang 17 33 0 01 May 2019
Dynamic Mini-batch SGD for Elastic Distributed Training: Learning in the Limbo of Resources Haibin Lin Hang Zhang Yifei Ma Tong He Zhi-Li Zhang Sheng Zha Mu Li 17 23 0 26 Apr 2019
A Deterministic Gradient-Based Approach to Avoid Saddle Points L. Kreusser Stanley J. Osher Bao Wang ODL 18 3 0 21 Jan 2019
SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator Cong Fang C. J. Li Zhouchen Lin Tong Zhang 33 567 0 04 Jul 2018
Stochastic Nested Variance Reduction for Nonconvex Optimization Dongruo Zhou Pan Xu Quanquan Gu 19 146 0 20 Jun 2018
Defending Against Saddle Point Attack in Byzantine-Robust Distributed Learning Dong Yin Yudong Chen K. Ramchandran Peter L. Bartlett FedML 21 97 0 14 Jun 2018
Escaping Saddles with Stochastic Gradients Hadi Daneshmand Jonas Köhler Aurélien Lucchi Thomas Hofmann 11 161 0 15 Mar 2018
Smoothed analysis for low-rank solutions to semidefinite programs in quadratic penalty form Srinadh Bhojanapalli Nicolas Boumal Prateek Jain Praneeth Netrapalli 14 42 0 01 Mar 2018
signSGD: Compressed Optimisation for Non-Convex Problems Jeremy Bernstein Yu-Xiang Wang Kamyar Azizzadenesheli Anima Anandkumar FedML ODL 19 1,016 0 13 Feb 2018
Global Convergence of Langevin Dynamics Based Algorithms for Nonconvex Optimization Pan Xu Jinghui Chen Difan Zou Quanquan Gu 24 200 0 20 Jul 2017
Stochastic Heavy Ball S. Gadat Fabien Panloup Sofiane Saadane 8 103 0 14 Sep 2016
A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights Weijie Su Stephen P. Boyd Emmanuel J. Candes 97 1,151 0 04 Mar 2015
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 175 1,185 0 30 Nov 2014