On the Quality of the Initial Basin in Overspecified Neural Networks

13 November 2015

Papers citing "On the Quality of the Initial Basin in Overspecified Neural Networks"

36 / 36 papers shown

Title
Gradient-Guided Annealing for Domain Generalization Aristotelis Ballas Christos Diou OOD 222 0 0 27 Feb 2025
Loss Landscape of Shallow ReLU-like Neural Networks: Stationary Points, Saddle Escape, and Network Embedding Zhengqing Wu Berfin Simsek Francois Ged ODL 48 0 0 08 Feb 2024
When Expressivity Meets Trainability: Fewer than $n$ Neurons Can Work Jiawei Zhang Yushun Zhang Mingyi Hong Ruoyu Sun Zhi-Quan Luo 29 10 0 21 Oct 2022
Deep Unfolding of the DBFB Algorithm with Application to ROI CT Imaging with Limited Angular Density Marion Savanier Émilie Chouzenoux J. Pesquet C. Riddell 32 16 0 27 Sep 2022
On the generalization of learning algorithms that do not converge N. Chandramoorthy Andreas Loukas Khashayar Gatmiry Stefanie Jegelka MLT 19 11 0 16 Aug 2022
Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization Landscape Devansh Bisla Jing Wang A. Choromańska 27 34 0 20 Jan 2022
Optimization-Based Separations for Neural Networks Itay Safran Jason D. Lee 185 14 0 04 Dec 2021
Landscape analysis for shallow neural networks: complete classification of critical points for affine target functions Patrick Cheridito Arnulf Jentzen Florian Rossmannek 24 10 0 19 Mar 2021
Learning Graph Neural Networks with Approximate Gradient Descent Qunwei Li Shaofeng Zou Leon Wenliang Zhong GNN 32 1 0 07 Dec 2020
Pruning Convolutional Filters using Batch Bridgeout Najeeb Khan Ian Stavness 28 3 0 23 Sep 2020
On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them Chen Liu Mathieu Salzmann Tao R. Lin Ryota Tomioka Sabine Süsstrunk AAML 24 81 0 15 Jun 2020
Non-convergence of stochastic gradient descent in the training of deep neural networks Patrick Cheridito Arnulf Jentzen Florian Rossmannek 14 37 0 12 Jun 2020
On the Power and Limitations of Random Features for Understanding Neural Networks Gilad Yehudai Ohad Shamir MLT 26 181 0 01 Apr 2019
Non-attracting Regions of Local Minima in Deep and Wide Neural Networks Henning Petzka C. Sminchisescu 29 9 0 16 Dec 2018
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du J. Lee Haochuan Li Liwei Wang Masayoshi Tomizuka ODL 44 1,125 0 09 Nov 2018
A Priori Estimates of the Population Risk for Two-layer Neural Networks Weinan E Chao Ma Lei Wu 29 130 0 15 Oct 2018
Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel Colin Wei J. Lee Qiang Liu Tengyu Ma 26 245 0 12 Oct 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 56 1,252 0 04 Oct 2018
Learning One-hidden-layer ReLU Networks via Gradient Descent Xiao Zhang Yaodong Yu Lingxiao Wang Quanquan Gu MLT 30 134 0 20 Jun 2018
Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach Ryo Karakida S. Akaho S. Amari FedML 47 140 0 04 Jun 2018
How Many Samples are Needed to Estimate a Convolutional or Recurrent Neural Network? S. Du Yining Wang Xiyu Zhai Sivaraman Balakrishnan Ruslan Salakhutdinov Aarti Singh SSL 21 57 0 21 May 2018
Improved Learning of One-hidden-layer Convolutional Neural Networks with Overlaps S. Du Surbhi Goel MLT 30 17 0 20 May 2018
The Loss Surface of XOR Artificial Neural Networks D. Mehta Xiaojun Zhao Edgar A. Bernal D. Wales 34 19 0 06 Apr 2018
A Walk with SGD Chen Xing Devansh Arpit Christos Tsirigotis Yoshua Bengio 27 118 0 24 Feb 2018
Gradient descent with identity initialization efficiently learns positive definite linear transformations by deep residual networks Peter L. Bartlett D. Helmbold Philip M. Long 36 116 0 16 Feb 2018
Fix your classifier: the marginal value of training the last weight layer Elad Hoffer Itay Hubara Daniel Soudry 35 101 0 14 Jan 2018
Visualizing the Loss Landscape of Neural Nets Hao Li Zheng Xu Gavin Taylor Christoph Studer Tom Goldstein 108 1,848 0 28 Dec 2017
Spurious Local Minima are Common in Two-Layer ReLU Neural Networks Itay Safran Ohad Shamir 40 262 0 24 Dec 2017
Recovery Guarantees for One-hidden-layer Neural Networks Kai Zhong Zhao Song Prateek Jain Peter L. Bartlett Inderjit S. Dhillon MLT 34 336 0 10 Jun 2017
The loss surface of deep and wide neural networks Quynh N. Nguyen Matthias Hein ODL 51 283 0 26 Apr 2017
Geometric deep learning: going beyond Euclidean data M. Bronstein Joan Bruna Yann LeCun Arthur Szlam P. Vandergheynst GNN 261 3,243 0 24 Nov 2016
Local minima in training of neural networks G. Swirszcz Wojciech M. Czarnecki Razvan Pascanu ODL 37 73 0 19 Nov 2016
Topology and Geometry of Half-Rectified Network Optimization C. Freeman Joan Bruna 19 233 0 04 Nov 2016
Convexified Convolutional Neural Networks Yuchen Zhang Percy Liang Martin J. Wainwright 26 64 0 04 Sep 2016
No bad local minima: Data independent training error guarantees for multilayer neural networks Daniel Soudry Y. Carmon 19 235 0 26 May 2016
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 183 1,185 0 30 Nov 2014