How to Escape Saddle Points Efficiently

2 March 2017

Papers citing "How to Escape Saddle Points Efficiently"

50 / 167 papers shown

Title
Noise Regularizes Over-parameterized Rank One Matrix Recovery, Provably Tianyi Liu Yan Li Enlu Zhou Tuo Zhao 38 1 0 07 Feb 2022
$Restarted Nonconvex Accelerated Gradient Descent: No More Polylogarithmic Factor in the $O(ε^{-7/4})$ Complexity$ Restarted Nonconvex Accelerated Gradient Descent: No More Polylogarithmic Factor in the $O(ε^{-7/4})$ Complexity Huan Li Zhouchen Lin 42 21 0 27 Jan 2022
Differentially Private Temporal Difference Learning with Stochastic Nonconvex-Strongly-Concave Optimization Canzhe Zhao Yanjie Ze Jing Dong Baoxiang Wang Shuai Li 47 4 0 25 Jan 2022
Stability Based Generalization Bounds for Exponential Family Langevin Dynamics A. Banerjee Tiancong Chen Xinyan Li Yingxue Zhou 34 8 0 09 Jan 2022
Over-Parametrized Matrix Factorization in the Presence of Spurious Stationary Points Armin Eftekhari 24 1 0 25 Dec 2021
Escape saddle points by a simple gradient-descent based algorithm Chenyi Zhang Tongyang Li ODL 28 15 0 28 Nov 2021
Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications Khaled B. Letaief Yuanming Shi Jianmin Lu Jianhua Lu 43 416 0 24 Nov 2021
Learning equilibria with personalized incentives in a class of nonmonotone games F. Fabiani Andrea Simonetto Paul Goulart 22 11 0 06 Nov 2021
Faster Perturbed Stochastic Gradient Methods for Finding Local Minima Zixiang Chen Dongruo Zhou Quanquan Gu 40 1 0 25 Oct 2021
On the Global Convergence of Gradient Descent for multi-layer ResNets in the mean-field regime Zhiyan Ding Shi Chen Qin Li S. Wright MLT AI4CE 41 11 0 06 Oct 2021
On the Estimation Bias in Double Q-Learning Zhizhou Ren Guangxiang Zhu Haotian Hu Beining Han Jian-Hai Chen Chongjie Zhang 24 17 0 29 Sep 2021
Concave Utility Reinforcement Learning with Zero-Constraint Violations Mridul Agarwal Qinbo Bai Vaneet Aggarwal 36 12 0 12 Sep 2021
Supervising the Decoder of Variational Autoencoders to Improve Scientific Utility Liyun Tu Austin Talbot Neil Gallagher David Carlson DRL 32 2 0 09 Sep 2021
Coordinate Descent Methods for DC Minimization: Optimality Conditions and Global Convergence Ganzhao Yuan 30 3 0 09 Sep 2021
Nonconvex Factorization and Manifold Formulations are Almost Equivalent in Low-rank Matrix Optimization Yuetian Luo Xudong Li Anru R. Zhang 30 9 0 03 Aug 2021
The loss landscape of deep linear neural networks: a second-order analysis E. M. Achour Franccois Malgouyres Sébastien Gerchinovitz ODL 24 9 0 28 Jul 2021
Distributed stochastic optimization with large delays Zhengyuan Zhou P. Mertikopoulos Nicholas Bambos Peter Glynn Yinyu Ye 20 9 0 06 Jul 2021
Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction Dominik Stöger Mahdi Soltanolkotabi ODL 42 75 0 28 Jun 2021
Global Convergence of Gradient Descent for Asymmetric Low-Rank Matrix Factorization Tian-Chun Ye S. Du 21 46 0 27 Jun 2021
A Survey on Fault-tolerance in Distributed Optimization and Machine Learning Shuo Liu AI4CE OOD 50 13 0 16 Jun 2021
Unique sparse decomposition of low rank matrices Dian Jin Xin Bing Yuqian Zhang 19 4 0 14 Jun 2021
Fast Federated Learning in the Presence of Arbitrary Device Unavailability Xinran Gu Kaixuan Huang Jingzhao Zhang Longbo Huang FedML 32 95 0 08 Jun 2021
Smooth Bilevel Programming for Sparse Regularization C. Poon Gabriel Peyré 11 18 0 02 Jun 2021
Escaping Saddle Points with Compressed SGD Dmitrii Avdiukhin G. Yaroslavtsev 19 4 0 21 May 2021
Sharp Restricted Isometry Property Bounds for Low-rank Matrix Recovery Problems with Corrupted Measurements Ziye Ma Yingjie Bi Javad Lavaei Somayeh Sojoudi 26 14 0 18 May 2021
Turning Channel Noise into an Accelerator for Over-the-Air Principal Component Analysis Zezhong Zhang Guangxu Zhu Rui-cang Wang Vincent K. N. Lau Kaibin Huang 33 31 0 20 Apr 2021
Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization Zeke Xie Li-xin Yuan Zhanxing Zhu Masashi Sugiyama 27 29 0 31 Mar 2021
On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs) Zhiyuan Li Sadhika Malladi Sanjeev Arora 44 78 0 24 Feb 2021
Noisy Gradient Descent Converges to Flat Minima for Nonconvex Matrix Factorization Tianyi Liu Yan Li S. Wei Enlu Zhou T. Zhao 21 13 0 24 Feb 2021
Stochastic Gradient Langevin Dynamics with Variance Reduction Zhishen Huang Stephen Becker 13 7 0 12 Feb 2021
Efficient Semi-Implicit Variational Inference Vincent Moens Hang Ren A. Maraval Rasul Tutunov Jun Wang H. Ammar 85 6 0 15 Jan 2021
The Nonconvex Geometry of Linear Inverse Problems Armin Eftekhari Peyman Mohajerin Esfahani 26 1 0 07 Jan 2021
Learning Graph Neural Networks with Approximate Gradient Descent Qunwei Li Shaofeng Zou Leon Wenliang Zhong GNN 32 1 0 07 Dec 2020
Regularized linear autoencoders recover the principal components, eventually Xuchan Bao James Lucas Sushant Sachdeva Roger C. Grosse 42 29 0 13 Jul 2020
Second-Order Information in Non-Convex Stochastic Optimization: Power and Limitations Yossi Arjevani Y. Carmon John C. Duchi Dylan J. Foster Ayush Sekhari Karthik Sridharan 84 53 0 24 Jun 2020
On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems P. Mertikopoulos Nadav Hallak Ali Kavis V. Cevher 21 85 0 19 Jun 2020
An Analysis of Constant Step Size SGD in the Non-convex Regime: Asymptotic Normality and Bias Lu Yu Krishnakumar Balasubramanian S. Volgushev Murat A. Erdogdu 35 50 0 14 Jun 2020
Evading Curse of Dimensionality in Unconstrained Private GLMs via Private Gradient Descent Shuang Song Thomas Steinke Om Thakkar Abhradeep Thakurta 27 50 0 11 Jun 2020
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning Z. Yao A. Gholami Sheng Shen Mustafa Mustafa Kurt Keutzer Michael W. Mahoney ODL 16 273 0 01 Jun 2020
Accelerating Ill-Conditioned Low-Rank Matrix Estimation via Scaled Gradient Descent Tian Tong Cong Ma Yuejie Chi 27 115 0 18 May 2020
Likelihood landscape and maximum likelihood estimation for the discrete orbit recovery model Z. Fan Yi Sun Tianhao Wang Yihong Wu 30 18 0 31 Mar 2020
Critical Point-Finding Methods Reveal Gradient-Flat Regions of Deep Network Losses Charles G. Frye James B. Simon Neha S. Wadia A. Ligeralde M. DeWeese K. Bouchard ODL 16 2 0 23 Mar 2020
Columnwise Element Selection for Computationally Efficient Nonnegative Coupled Matrix Tensor Factorization Thirunavukarasu Balasubramaniam R. Nayak Chau Yuen 8 7 0 07 Mar 2020
Adaptive Federated Optimization Sashank J. Reddi Zachary B. Charles Manzil Zaheer Zachary Garrett Keith Rush Jakub Konecný Sanjiv Kumar H. B. McMahan FedML 20 1,391 0 29 Feb 2020
First Order Methods take Exponential Time to Converge to Global Minimizers of Non-Convex Functions Krishna Reddy Kesari Jean Honorio 6 1 0 28 Feb 2020
The Landscape of Matrix Factorization Revisited Hossein Valavi Sulin Liu Peter J. Ramadge 14 5 0 27 Feb 2020
Low Rank Saddle Free Newton: A Scalable Method for Stochastic Nonconvex Optimization Thomas O'Leary-Roseberry Nick Alger Omar Ghattas ODL 37 9 0 07 Feb 2020
On the Sample Complexity and Optimization Landscape for Quadratic Feasibility Problems Parth Thaker Gautam Dasarathy Angelia Nedić 24 10 0 04 Feb 2020
Replica Exchange for Non-Convex Optimization Jing-rong Dong Xin T. Tong 22 21 0 23 Jan 2020
Intermittent Pulling with Local Compensation for Communication-Efficient Federated Learning Yining Qi Zhihao Qu Song Guo Xin Gao Ruixuan Li Baoliu Ye FedML 18 8 0 22 Jan 2020