Identifying and attacking the saddle point problem in high-dimensional non-convex optimization

10 June 2014

Papers citing "Identifying and attacking the saddle point problem in high-dimensional non-convex optimization"

50 / 251 papers shown

Title
Measuring the Intrinsic Dimension of Objective Landscapes Chunyuan Li Heerad Farkhoor Rosanne Liu J. Yosinski 38 401 0 24 Apr 2018
On Gradient-Based Learning in Continuous Games Eric Mazumdar Lillian J. Ratliff S. Shankar Sastry 27 134 0 16 Apr 2018
The Loss Surface of XOR Artificial Neural Networks D. Mehta Xiaojun Zhao Edgar A. Bernal D. Wales 34 19 0 06 Apr 2018
DeepSigns: A Generic Watermarking Framework for IP Protection of Deep Learning Models B. Rouhani Huili Chen F. Koushanfar 43 48 0 02 Apr 2018
A Survey on Deep Learning Methods for Robot Vision Javier Ruiz-del-Solar P. Loncomilla Naiomi Soto 31 60 0 28 Mar 2018
Comparing Dynamics: Deep Neural Networks versus Glassy Systems Marco Baity-Jesi Levent Sagun Mario Geiger S. Spigler Gerard Ben Arous C. Cammarota Yann LeCun M. Wyart Giulio Biroli AI4CE 42 113 0 19 Mar 2018
Replica Symmetry Breaking in Bipartite Spin Glasses and Neural Networks Gavin Hartnett Edward Parker Edward Geist 21 23 0 17 Mar 2018
Escaping Saddles with Stochastic Gradients Hadi Daneshmand Jonas Köhler Aurelien Lucchi Thomas Hofmann 27 162 0 15 Mar 2018
Accelerating Natural Gradient with Higher-Order Invariance Yang Song Jiaming Song Stefano Ermon 26 21 0 04 Mar 2018
Essentially No Barriers in Neural Network Energy Landscape Felix Dräxler K. Veschgini M. Salmhofer Fred Hamprecht MoMe 22 424 0 02 Mar 2018
Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs T. Garipov Pavel Izmailov Dmitrii Podoprikhin Dmitry Vetrov A. Wilson UQCV 27 734 0 27 Feb 2018
A Walk with SGD Chen Xing Devansh Arpit Christos Tsirigotis Yoshua Bengio 27 118 0 24 Feb 2018
Understanding the Loss Surface of Neural Networks for Binary Classification Shiyu Liang Ruoyu Sun Yixuan Li R. Srikant 35 87 0 19 Feb 2018
The Mechanics of n-Player Differentiable Games David Balduzzi S. Racanière James Martens Jakob N. Foerster K. Tuyls T. Graepel MLT 30 274 0 15 Feb 2018
Deep Neural Networks Learn Non-Smooth Functions Effectively Masaaki Imaizumi Kenji Fukumizu 26 123 0 13 Feb 2018
signSGD: Compressed Optimisation for Non-Convex Problems Jeremy Bernstein Yu Wang Kamyar Azizzadenesheli Anima Anandkumar FedML ODL 44 1,021 0 13 Feb 2018
Digital Watermarking for Deep Neural Networks Yuki Nagai Yusuke Uchida S. Sakazawa Shiníchi Satoh WIGM 31 144 0 06 Feb 2018
Structured Inhomogeneous Density Map Learning for Crowd Counting Hanhui Li Xiangjian He Hefeng Wu Saeed Amirgholipour Kasmani Ruomei Wang Xiaonan Luo Liang Lin 30 12 0 20 Jan 2018
Near Maximum Likelihood Decoding with Deep Learning Eliya Nachmani Yaron Bachar Elad Marciano D. Burshtein Yair Be’ery 27 25 0 08 Jan 2018
High Dimensional Spaces, Deep Learning and Adversarial Examples S. Dube 37 29 0 02 Jan 2018
Accelerating Deep Learning with Memcomputing Haik Manukian F. Traversa M. Di Ventra AI4CE 33 33 0 01 Jan 2018
Visualizing the Loss Landscape of Neural Nets Hao Li Zheng Xu Gavin Taylor Christoph Studer Tom Goldstein 114 1,850 0 28 Dec 2017
Non-convex Optimization for Machine Learning Prateek Jain Purushottam Kar 33 479 0 21 Dec 2017
Accelerated Gradient Descent Escapes Saddle Points Faster than Gradient Descent Chi Jin Praneeth Netrapalli Michael I. Jordan ODL 37 261 0 28 Nov 2017
Convergent Block Coordinate Descent for Training Tikhonov Regularized Deep Neural Networks Ziming Zhang M. Brand 26 70 0 20 Nov 2017
Neon2: Finding Local Minima via First-Order Oracles Zeyuan Allen-Zhu Yuanzhi Li 21 130 0 17 Nov 2017
Rethinking generalization requires revisiting old ideas: statistical mechanics approaches and complex learning behavior Charles H. Martin Michael W. Mahoney AI4CE 30 63 0 26 Oct 2017
AdaDNNs: Adaptive Ensemble of Deep Neural Networks for Scene Text Recognition Chun Yang Xu-Cheng Yin Zejun Li Jianwei Wu Chunchao Guo Hongfa Wang Lei Xiao 24 10 0 10 Oct 2017
Natasha 2: Faster Non-Convex Optimization Than SGD Zeyuan Allen-Zhu ODL 28 245 0 29 Aug 2017
Optimization Beyond the Convolution: Generalizing Spatial Relations with End-to-End Metric Learning P. Jund Andreas Eitel N. Abdo Wolfram Burgard 3DPC 24 19 0 04 Jul 2017
Optimization Methods for Supervised Machine Learning: From Linear Models to Deep Learning Frank E. Curtis K. Scheinberg 39 45 0 30 Jun 2017
Stochastic Training of Neural Networks via Successive Convex Approximations Simone Scardapane P. Di Lorenzo 24 9 0 15 Jun 2017
Proximal Backpropagation Thomas Frerix Thomas Möllenhoff Michael Möller Daniel Cremers 23 31 0 14 Jun 2017
A Well-Tempered Landscape for Non-convex Robust Subspace Recovery Tyler Maunu Teng Zhang Gilad Lerman 24 63 0 13 Jun 2017
Recovery Guarantees for One-hidden-layer Neural Networks Kai Zhong Zhao Song Prateek Jain Peter L. Bartlett Inderjit S. Dhillon MLT 34 336 0 10 Jun 2017
Global Convergence of the (1+1) Evolution Strategy Tobias Glasmachers 17 9 0 09 Jun 2017
Are Saddles Good Enough for Deep Learning? Adepu Ravi Sankar V. Balasubramanian 43 5 0 07 Jun 2017
Spectral Norm Regularization for Improving the Generalizability of Deep Learning Yuichi Yoshida Takeru Miyato 40 325 0 31 May 2017
Classification regions of deep neural networks Alhussein Fawzi Seyed-Mohsen Moosavi-Dezfooli P. Frossard Stefano Soatto 31 51 0 26 May 2017
Train longer, generalize better: closing the generalization gap in large batch training of neural networks Elad Hoffer Itay Hubara Daniel Soudry ODL 44 795 0 24 May 2017
Sub-sampled Cubic Regularization for Non-convex Optimization Jonas Köhler Aurelien Lucchi 19 165 0 16 May 2017
Deep neural networks on graph signals for brain imaging analysis Yiluan Guo Hossein Nejati Ngai-man Cheung GNN 19 26 0 13 May 2017
The loss surface of deep and wide neural networks Quynh N. Nguyen Matthias Hein ODL 51 283 0 26 Apr 2017
Deep Relaxation: partial differential equations for optimizing deep neural networks Pratik Chaudhari Adam M. Oberman Stanley Osher Stefano Soatto G. Carlier 27 153 0 17 Apr 2017
Snapshot Ensembles: Train 1, get M for free Gao Huang Yixuan Li Geoff Pleiss Zhuang Liu J. Hopcroft Kilian Q. Weinberger OOD FedML UQCV 50 935 0 01 Apr 2017
Failures of Gradient-Based Deep Learning Shai Shalev-Shwartz Ohad Shamir Shaked Shammah ODL UQCV 34 198 0 23 Mar 2017
Sharp Minima Can Generalize For Deep Nets Laurent Dinh Razvan Pascanu Samy Bengio Yoshua Bengio ODL 46 758 0 15 Mar 2017
Langevin Dynamics with Continuous Tempering for Training Deep Neural Networks Nanyang Ye Zhanxing Zhu Rafał K. Mantiuk 21 20 0 13 Mar 2017
How to Escape Saddle Points Efficiently Chi Jin Rong Ge Praneeth Netrapalli Sham Kakade Michael I. Jordan ODL 37 832 0 02 Mar 2017
On the Origin of Deep Learning Haohan Wang Bhiksha Raj MedIm 3DV VLM 48 223 0 24 Feb 2017