How to Escape Saddle Points Efficiently

2 March 2017

Papers citing "How to Escape Saddle Points Efficiently"

50 / 167 papers shown

Title
Analysis of the Optimization Landscapes for Overcomplete Representation Learning Qing Qu Yuexiang Zhai Xiao Li Yuqian Zhang Zhihui Zhu 20 9 0 05 Dec 2019
Online Stochastic Gradient Descent with Arbitrary Initialization Solves Non-smooth, Non-convex Phase Retrieval Yan Shuo Tan Roman Vershynin 22 35 0 28 Oct 2019
Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networks Yu Bai J. Lee 24 116 0 03 Oct 2019
Quantum algorithm for finding the negative curvature direction in non-convex optimization Kaining Zhang Min-hsiu Hsieh Liu Liu Dacheng Tao 13 3 0 17 Sep 2019
Meta-Learning with Implicit Gradients Aravind Rajeswaran Chelsea Finn Sham Kakade Sergey Levine 19 844 0 10 Sep 2019
Short-and-Sparse Deconvolution -- A Geometric Approach Yenson Lau Qing Qu Han-Wen Kuo Pengcheng Zhou Yuqian Zhang John N. Wright 17 29 0 28 Aug 2019
Theoretical Issues in Deep Networks: Approximation, Optimization and Generalization T. Poggio Andrzej Banburski Q. Liao ODL 29 161 0 25 Aug 2019
Second-Order Guarantees of Stochastic Gradient Descent in Non-Convex Optimization Stefan Vlaski Ali H. Sayed ODL 23 21 0 19 Aug 2019
Instance Enhancement Batch Normalization: an Adaptive Regulator of Batch Noise Senwei Liang Zhongzhan Huang Mingfu Liang Haizhao Yang 30 57 0 12 Aug 2019
On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shift Alekh Agarwal Sham Kakade J. Lee G. Mahajan 11 316 0 01 Aug 2019
Hyperlink Regression via Bregman Divergence Akifumi Okuno Hidetoshi Shimodaira 33 6 0 22 Jul 2019
Distributed Inexact Successive Convex Approximation ADMM: Analysis-Part I Sandeep Kumar K. Rajawat Daniel P. Palomar 19 4 0 21 Jul 2019
SNAP: Finding Approximate Second-Order Stationary Solutions Efficiently for Non-convex Linearly Constrained Problems Songtao Lu Meisam Razaviyayn Bo Yang Kejun Huang Mingyi Hong 27 11 0 09 Jul 2019
Distributed Learning in Non-Convex Environments -- Part II: Polynomial Escape from Saddle-Points Stefan Vlaski Ali H. Sayed 21 53 0 03 Jul 2019
Global Convergence of Policy Gradient Methods to (Almost) Locally Optimal Policies Kaipeng Zhang Alec Koppel Haoqi Zhu Tamer Basar 41 186 0 19 Jun 2019
On the Noisy Gradient Descent that Generalizes as SGD Jingfeng Wu Wenqing Hu Haoyi Xiong Jun Huan Vladimir Braverman Zhanxing Zhu MLT 24 10 0 18 Jun 2019
Global Optimality Guarantees For Policy Gradient Methods Jalaj Bhandari Daniel Russo 37 185 0 05 Jun 2019
Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems Tianle Cai Ruiqi Gao Jikai Hou Siyu Chen Dong Wang Di He Zhihua Zhang Liwei Wang ODL 21 57 0 28 May 2019
Stochastic Iterative Hard Thresholding for Graph-structured Sparsity Optimization Baojian Zhou F. Chen Yiming Ying 26 7 0 09 May 2019
Stabilized SVRG: Simple Variance Reduction for Nonconvex Optimization Rong Ge Zhize Li Weiyao Wang Xiang Wang 19 33 0 01 May 2019
Annealing for Distributed Global Optimization Brian Swenson S. Kar H. Vincent Poor J. M. F. Moura 25 30 0 18 Mar 2019
An Empirical Study of Large-Batch Stochastic Gradient Descent with Structured Covariance Noise Yeming Wen Kevin Luk Maxime Gazeau Guodong Zhang Harris Chan Jimmy Ba ODL 20 22 0 21 Feb 2019
Asymmetric Valleys: Beyond Sharp and Flat Local Minima Haowei He Gao Huang Yang Yuan ODL MLT 22 147 0 02 Feb 2019
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruosong Wang MLT 37 961 0 24 Jan 2019
Width Provably Matters in Optimization for Deep Linear Neural Networks S. Du Wei Hu 21 94 0 24 Jan 2019
A Deterministic Gradient-Based Approach to Avoid Saddle Points L. Kreusser Stanley J. Osher Bao Wang ODL 29 3 0 21 Jan 2019
SGD Converges to Global Minimum in Deep Learning via Star-convex Path Yi Zhou Junjie Yang Huishuai Zhang Yingbin Liang Vahid Tarokh 14 71 0 02 Jan 2019
Blind Over-the-Air Computation and Data Fusion via Provable Wirtinger Flow Jialin Dong Yuanming Shi Z. Ding 9 59 0 12 Nov 2018
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du J. Lee Haochuan Li Liwei Wang Masayoshi Tomizuka ODL 41 1,122 0 09 Nov 2018
Understanding the Acceleration Phenomenon via High-Resolution Differential Equations Bin Shi S. Du Michael I. Jordan Weijie J. Su 17 254 0 21 Oct 2018
Fault Tolerance in Iterative-Convergent Machine Learning Aurick Qiao Bryon Aragam Bingjing Zhang Eric P. Xing 26 41 0 17 Oct 2018
Stein Neural Sampler Tianyang Hu Zixiang Chen Hanxi Sun Jincheng Bai Mao Ye Guang Cheng SyDa GAN 22 34 0 08 Oct 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 38 1,250 0 04 Oct 2018
Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile P. Mertikopoulos Bruno Lecouat Houssam Zenati Chuan-Sheng Foo V. Chandrasekhar Georgios Piliouras 32 291 0 07 Jul 2018
SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator Cong Fang C. J. Li Zhouchen Lin Tong Zhang 44 570 0 04 Jul 2018
On the Implicit Bias of Dropout Poorya Mianjy R. Arora René Vidal 27 66 0 26 Jun 2018
Stochastic Nested Variance Reduction for Nonconvex Optimization Dongruo Zhou Pan Xu Quanquan Gu 25 146 0 20 Jun 2018
Learning One-hidden-layer ReLU Networks via Gradient Descent Xiao Zhang Yaodong Yu Lingxiao Wang Quanquan Gu MLT 28 134 0 20 Jun 2018
Defending Against Saddle Point Attack in Byzantine-Robust Distributed Learning Dong Yin Yudong Chen Kannan Ramchandran Peter L. Bartlett FedML 29 97 0 14 Jun 2018
Structured Local Optima in Sparse Blind Deconvolution Yuqian Zhang Han-Wen Kuo John N. Wright 24 56 0 01 Jun 2018
Improved Learning of One-hidden-layer Convolutional Neural Networks with Overlaps S. Du Surbhi Goel MLT 25 17 0 20 May 2018
The Global Optimization Geometry of Shallow Linear Neural Networks Zhihui Zhu Daniel Soudry Yonina C. Eldar M. Wakin ODL 18 36 0 13 May 2018
On Gradient-Based Learning in Continuous Games Eric Mazumdar Lillian J. Ratliff S. Shankar Sastry 8 134 0 16 Apr 2018
Tight Query Complexity Lower Bounds for PCA via Finite Sample Deformed Wigner Law Max Simchowitz A. Alaoui Benjamin Recht 27 38 0 04 Apr 2018
Bayesian Optimization with Expensive Integrands Saul Toscano-Palmerin P. Frazier 13 49 0 23 Mar 2018
Escaping Saddles with Stochastic Gradients Hadi Daneshmand Jonas Köhler Aurelien Lucchi Thomas Hofmann 21 161 0 15 Mar 2018
Fast and Sample Efficient Inductive Matrix Completion via Multi-Phase Procrustes Flow Xiao Zhang S. Du Quanquan Gu 26 24 0 03 Mar 2018
Smoothed analysis for low-rank solutions to semidefinite programs in quadratic penalty form Srinadh Bhojanapalli Nicolas Boumal Prateek Jain Praneeth Netrapalli 23 42 0 01 Mar 2018
Stochastic Variance-Reduced Cubic Regularization for Nonconvex Optimization Zhe Wang Yi Zhou Yingbin Liang Guanghui Lan 35 46 0 20 Feb 2018
Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator Maryam Fazel Rong Ge Sham Kakade M. Mesbahi 35 598 0 15 Jan 2018