Elimination of All Bad Local Minima in Deep Learning

2 January 2019

Papers citing "Elimination of All Bad Local Minima in Deep Learning"

31 / 31 papers shown

Title
Gradient Descent Finds Global Minima for Generalizable Deep Neural Networks of Practical Sizes Kenji Kawaguchi Jiaoyang Huang ODL 11 57 0 05 Aug 2019
An Improved Analysis of Training Over-parameterized Deep Neural Networks Difan Zou Quanquan Gu 39 230 0 11 Jun 2019
Every Local Minimum Value is the Global Minimum Value of Induced Model in Non-convex Machine Learning Kenji Kawaguchi Jiaoyang Huang L. Kaelbling AAML 49 18 0 07 Apr 2019
Width Provably Matters in Optimization for Deep Linear Neural Networks S. Du Wei Hu 44 94 0 24 Jan 2019
Deep Learning for Classical Japanese Literature Tarin Clanuwat Mikel Bober-Irizar A. Kitamoto Alex Lamb Kazuaki Yamamoto David R Ha 57 705 0 03 Dec 2018
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks Difan Zou Yuan Cao Dongruo Zhou Quanquan Gu ODL 55 448 0 21 Nov 2018
Effect of Depth and Width on Local Minima in Deep Learning Kenji Kawaguchi Jiaoyang Huang L. Kaelbling 24 55 0 20 Nov 2018
A Convergence Theory for Deep Learning via Over-Parameterization Zeyuan Allen-Zhu Yuanzhi Li Zhao Song AI4CE ODL 94 1,457 0 09 Nov 2018
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du Jason D. Lee Haochuan Li Liwei Wang Masayoshi Tomizuka ODL 61 1,133 0 09 Nov 2018
Depth with Nonlinearity Creates No Bad Local Minima in ResNets Kenji Kawaguchi Yoshua Bengio ODL 54 64 0 21 Oct 2018
On the loss landscape of a class of deep neural networks with no bad local valleys Quynh N. Nguyen Mahesh Chandra Mukkamala Matthias Hein 36 87 0 27 Sep 2018
Learning ReLU Networks on Linearly Separable Data: Algorithm, Optimality, and Generalization G. Wang G. Giannakis Jie Chen MLT 34 131 0 14 Aug 2018
Deep Neural Networks with Multi-Branch Architectures Are Less Non-Convex Hongyang R. Zhang Junru Shao Ruslan Salakhutdinov 53 14 0 06 Jun 2018
Adding One Neuron Can Eliminate All Bad Local Minima Shiyu Liang Ruoyu Sun Jason D. Lee R. Srikant 59 89 0 22 May 2018
Are ResNets Provably Better than Linear Predictors? Ohad Shamir 19 54 0 18 Apr 2018
On the Power of Over-parametrization in Neural Networks with Quadratic Activation S. Du Jason D. Lee 82 268 0 03 Mar 2018
Gradient descent with identity initialization efficiently learns positive definite linear transformations by deep residual networks Peter L. Bartlett D. Helmbold Philip M. Long 53 116 0 16 Feb 2018
Deep linear neural networks with arbitrary loss: All local minima are global T. Laurent J. V. Brecht ODL 12 136 0 05 Dec 2017
Learning One-hidden-layer Neural Networks with Landscape Design Rong Ge Jason D. Lee Tengyu Ma MLT 46 260 0 01 Nov 2017
Learning Neural Networks with Two Nonlinear Layers in Polynomial Time Surbhi Goel Adam R. Klivans 29 52 0 18 Sep 2017
Recovery Guarantees for One-hidden-layer Neural Networks Kai Zhong Zhao Song Prateek Jain Peter L. Bartlett Inderjit S. Dhillon MLT 53 336 0 10 Jun 2017
Convergence Analysis of Two-layer Neural Networks with ReLU Activation Yuanzhi Li Yang Yuan MLT 18 649 0 28 May 2017
Learning ReLUs via Gradient Descent Mahdi Soltanolkotabi MLT 48 181 0 10 May 2017
The loss surface of deep and wide neural networks Quynh N. Nguyen Matthias Hein ODL 70 284 0 26 Apr 2017
Globally Optimal Gradient Descent for a ConvNet with Gaussian Inputs Alon Brutzkus Amir Globerson MLT 26 313 0 26 Feb 2017
Exponentially vanishing sub-optimal local minima in multilayer neural networks Daniel Soudry Elad Hoffer 59 97 0 19 Feb 2017
Identity Matters in Deep Learning Moritz Hardt Tengyu Ma OOD 34 398 0 14 Nov 2016
Deep Learning without Poor Local Minima Kenji Kawaguchi ODL 55 919 0 23 May 2016
Bayesian Optimization with Exponential Convergence Kenji Kawaguchi L. Kaelbling Tomás Lozano-Pérez 40 106 0 05 Apr 2016
Provable Methods for Training Neural Networks with Sparse Connectivity Hanie Sedghi Anima Anandkumar 32 64 0 08 Dec 2014
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 208 1,189 0 30 Nov 2014