Towards moderate overparameterization: global convergence guarantees for training shallow neural networks

12 February 2019

Papers citing "Towards moderate overparameterization: global convergence guarantees for training shallow neural networks"

31 / 131 papers shown

Title
Optimization for deep learning: theory and algorithms Ruoyu Sun ODL 137 169 0 19 Dec 2019
Towards Understanding the Spectral Bias of Deep Learning Yuan Cao Zhiying Fang Yue Wu Ding-Xuan Zhou Quanquan Gu 116 219 0 03 Dec 2019
How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks? Zixiang Chen Yuan Cao Difan Zou Quanquan Gu 77 123 0 27 Nov 2019
Sub-Optimal Local Minima Exist for Neural Networks with Almost All Non-Linear Activations Tian Ding Dawei Li Ruoyu Sun 88 13 0 04 Nov 2019
Denoising and Regularization via Exploiting the Structural Bias of Convolutional Generators Reinhard Heckel Mahdi Soltanolkotabi DiffM 104 83 0 31 Oct 2019
Active Subspace of Neural Networks: Structural Analysis and Universal Attacks Chunfeng Cui Kaiqi Zhang Talgat Daulbaev Julia Gusak Ivan Oseledets Zheng Zhang AAML 61 25 0 29 Oct 2019
The Local Elasticity of Neural Networks Hangfeng He Weijie J. Su 149 46 0 15 Oct 2019
Nearly Minimal Over-Parametrization of Shallow Neural Networks Armin Eftekhari Chaehwan Song Volkan Cevher 45 1 0 09 Oct 2019
Polylogarithmic width suffices for gradient descent to achieve arbitrarily small test error with shallow ReLU networks Ziwei Ji Matus Telgarsky 98 178 0 26 Sep 2019
Effect of Activation Functions on the Training of Overparametrized Neural Nets A. Panigrahi Abhishek Shetty Navin Goyal 81 21 0 16 Aug 2019
The generalization error of random features regression: Precise asymptotics and double descent curve Song Mei Andrea Montanari 134 640 0 14 Aug 2019
Trainability of ReLU networks and Data-dependent Initialization Yeonjong Shin George Karniadakis 40 8 0 23 Jul 2019
Algorithmic Guarantees for Inverse Imaging with Untrained Network Priors Gauri Jagatap Chinmay Hegde 89 74 0 20 Jun 2019
ID3 Learns Juntas for Smoothed Product Distributions Alon Brutzkus Amit Daniely Eran Malach 67 21 0 20 Jun 2019
Approximation power of random neural networks Bolton Bailey Ziwei Ji Matus Telgarsky Ruicheng Xian 55 6 0 18 Jun 2019
Generalization Guarantees for Neural Networks via Harnessing the Low-rank Structure of the Jacobian Samet Oymak Zalan Fabian Mingchen Li Mahdi Soltanolkotabi MLT 87 88 0 12 Jun 2019
Decoupling Gating from Linearity Jonathan Fiat Eran Malach Shai Shalev-Shwartz 126 28 0 12 Jun 2019
An Improved Analysis of Training Over-parameterized Deep Neural Networks Difan Zou Quanquan Gu 74 235 0 11 Jun 2019
Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks Yuan Cao Quanquan Gu MLT AI4CE 110 392 0 30 May 2019
Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks Guodong Zhang James Martens Roger C. Grosse ODL 113 126 0 27 May 2019
Temporal-difference learning with nonlinear function approximation: lazy training and mean field regimes Andrea Agazzi Jianfeng Lu 71 8 0 27 May 2019
On Learning Over-parameterized Neural Networks: A Functional Approximation Perspective Lili Su Pengkun Yang MLT 75 54 0 26 May 2019
Gradient Descent can Learn Less Over-parameterized Two-layer Neural Networks on Classification Problems Atsushi Nitanda Geoffrey Chinot Taiji Suzuki MLT 105 34 0 23 May 2019
Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks Mingchen Li Mahdi Soltanolkotabi Samet Oymak NoLa 134 355 0 27 Mar 2019
On the Benefit of Width for Neural Networks: Disappearance of Bad Basins Dawei Li Tian Ding Ruoyu Sun 114 38 0 28 Dec 2018
On Lazy Training in Differentiable Programming Lénaïc Chizat Edouard Oyallon Francis R. Bach 111 840 0 19 Dec 2018
Piecewise Strong Convexity of Neural Networks Tristan Milne 60 21 0 30 Oct 2018
Learning ReLU Networks on Linearly Separable Data: Algorithm, Optimality, and Generalization G. Wang G. Giannakis Jie Chen MLT 83 131 0 14 Aug 2018
Compressed Sensing with Deep Image Prior and Learned Regularization Dave Van Veen A. Jalal Mahdi Soltanolkotabi Eric Price S. Vishwanath A. Dimakis 137 182 0 17 Jun 2018
Spurious Valleys in Two-layer Neural Network Optimization Landscapes Luca Venturi Afonso S. Bandeira Joan Bruna 82 74 0 18 Feb 2018
Global Guarantees for Enforcing Deep Generative Priors by Empirical Risk Paul Hand V. Voroninski UQCV 160 138 0 22 May 2017