Gradient Descent Provably Optimizes Over-parameterized Neural Networks

4 October 2018

Aarti Singh

Papers citing "Gradient Descent Provably Optimizes Over-parameterized Neural Networks"

44 / 244 papers shown

Title
Neural Tangents: Fast and Easy Infinite Neural Networks in Python Roman Novak Lechao Xiao Jiri Hron Jaehoon Lee Alexander A. Alemi Jascha Narain Sohl-Dickstein S. Schoenholz 27 224 0 05 Dec 2019
Towards Understanding the Spectral Bias of Deep Learning Yuan Cao Zhiying Fang Yue Wu Ding-Xuan Zhou Quanquan Gu 21 214 0 03 Dec 2019
Neural Contextual Bandits with UCB-based Exploration Dongruo Zhou Lihong Li Quanquan Gu 17 15 0 11 Nov 2019
Enhanced Convolutional Neural Tangent Kernels Zhiyuan Li Ruosong Wang Dingli Yu S. Du Wei Hu Ruslan Salakhutdinov Sanjeev Arora 16 131 0 03 Nov 2019
Global Convergence of Gradient Descent for Deep Linear Residual Networks Lei Wu Qingcan Wang Chao Ma ODL AI4CE 20 22 0 02 Nov 2019
Denoising and Regularization via Exploiting the Structural Bias of Convolutional Generators Reinhard Heckel Mahdi Soltanolkotabi DiffM 27 81 0 31 Oct 2019
Capacity, Bandwidth, and Compositionality in Emergent Language Learning Cinjon Resnick Abhinav Gupta Jakob N. Foerster Andrew M. Dai Kyunghyun Cho 18 51 0 24 Oct 2019
Harnessing the Power of Infinitely Wide Deep Nets on Small-data Tasks Sanjeev Arora S. Du Zhiyuan Li Ruslan Salakhutdinov Ruosong Wang Dingli Yu AAML 9 161 0 03 Oct 2019
Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networks Yu Bai J. Lee 11 116 0 03 Oct 2019
Overparameterized Neural Networks Implement Associative Memory Adityanarayanan Radhakrishnan M. Belkin Caroline Uhler BDL 19 71 0 26 Sep 2019
Towards Understanding the Transferability of Deep Representations Hong Liu Mingsheng Long Jianmin Wang Michael I. Jordan 16 25 0 26 Sep 2019
Asymptotics of Wide Networks from Feynman Diagrams Ethan Dyer Guy Gur-Ari 19 113 0 25 Sep 2019
Sample Efficient Policy Gradient Methods with Recursive Variance Reduction Pan Xu F. Gao Quanquan Gu 23 83 0 18 Sep 2019
The generalization error of random features regression: Precise asymptotics and double descent curve Song Mei Andrea Montanari 39 624 0 14 Aug 2019
Kernel and Rich Regimes in Overparametrized Models Blake E. Woodworth Suriya Gunasekar Pedro H. P. Savarese E. Moroshko Itay Golan J. Lee Daniel Soudry Nathan Srebro 19 353 0 13 Jun 2019
Generalization bounds for deep convolutional neural networks Philip M. Long Hanie Sedghi MLT 34 89 0 29 May 2019
Norm-based generalisation bounds for multi-class convolutional neural networks Antoine Ledent Waleed Mustafa Yunwen Lei Marius Kloft 8 5 0 29 May 2019
Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems Tianle Cai Ruiqi Gao Jikai Hou Siyu Chen Dong Wang Di He Zhihua Zhang Liwei Wang ODL 11 57 0 28 May 2019
Enhancing Adversarial Defense by k-Winners-Take-All Chang Xiao Peilin Zhong Changxi Zheng AAML 11 97 0 25 May 2019
What Can ResNet Learn Efficiently, Going Beyond Kernels? Zeyuan Allen-Zhu Yuanzhi Li 24 183 0 24 May 2019
Gradient Descent can Learn Less Over-parameterized Two-layer Neural Networks on Classification Problems Atsushi Nitanda Geoffrey Chinot Taiji Suzuki MLT 8 33 0 23 May 2019
A type of generalization error induced by initialization in deep neural networks Yaoyu Zhang Zhi-Qin John Xu Tao Luo Zheng Ma 9 49 0 19 May 2019
Linearized two-layers neural networks in high dimension Behrooz Ghorbani Song Mei Theodor Misiakiewicz Andrea Montanari MLT 11 241 0 27 Apr 2019
On Exact Computation with an Infinitely Wide Neural Net Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruslan Salakhutdinov Ruosong Wang 22 899 0 26 Apr 2019
Analysis of the Gradient Descent Algorithm for a Deep Neural Network Model with Skip-connections E. Weinan Chao Ma Qingcan Wang Lei Wu MLT 21 22 0 10 Apr 2019
Correlation Congruence for Knowledge Distillation Baoyun Peng Xiao Jin Jiaheng Liu Shunfeng Zhou Yichao Wu Yu Liu Dongsheng Li Zhaoning Zhang 37 507 0 03 Apr 2019
Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks Mingchen Li Mahdi Soltanolkotabi Samet Oymak NoLa 26 350 0 27 Mar 2019
A Priori Estimates of the Population Risk for Residual Networks E. Weinan Chao Ma Qingcan Wang UQCV 15 61 0 06 Mar 2019
Understanding over-parameterized deep networks by geometrization Xiao Dong Ling Zhou GNN AI4CE 13 7 0 11 Feb 2019
Stiffness: A New Perspective on Generalization in Neural Networks Stanislav Fort Pawel Krzysztof Nowak Stanislaw Jastrzebski S. Narayanan 19 94 0 28 Jan 2019
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruosong Wang MLT 29 961 0 24 Jan 2019
Width Provably Matters in Optimization for Deep Linear Neural Networks S. Du Wei Hu 13 93 0 24 Jan 2019
Scaling description of generalization with number of parameters in deep learning Mario Geiger Arthur Jacot S. Spigler Franck Gabriel Levent Sagun Stéphane dÁscoli Giulio Biroli Clément Hongler M. Wyart 33 194 0 06 Jan 2019
Analysis of a Two-Layer Neural Network via Displacement Convexity Adel Javanmard Marco Mondelli Andrea Montanari MLT 40 57 0 05 Jan 2019
Deep Geometric Prior for Surface Reconstruction Francis Williams T. Schneider Claudio Silva Denis Zorin Joan Bruna Daniele Panozzo 3DPC 14 189 0 27 Nov 2018
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks Difan Zou Yuan Cao Dongruo Zhou Quanquan Gu ODL 11 446 0 21 Nov 2018
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du J. Lee Haochuan Li Liwei Wang M. Tomizuka ODL 15 1,120 0 09 Nov 2018
Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity Chulhee Yun S. Sra Ali Jadbabaie 13 117 0 17 Oct 2018
Learning Two-layer Neural Networks with Symmetric Inputs Rong Ge Rohith Kuditipudi Zhize Li Xiang Wang OOD MLT 25 57 0 16 Oct 2018
A Priori Estimates of the Population Risk for Two-layer Neural Networks Weinan E Chao Ma Lei Wu 24 130 0 15 Oct 2018
Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel Colin Wei J. Lee Qiang Liu Tengyu Ma 16 243 0 12 Oct 2018
Unbiased deep solvers for linear parametric PDEs Marc Sabate Vidales David Siska Lukasz Szpruch OOD 24 7 0 11 Oct 2018
Learning One-hidden-layer ReLU Networks via Gradient Descent Xiao Zhang Yaodong Yu Lingxiao Wang Quanquan Gu MLT 24 134 0 20 Jun 2018
Compressed Sensing with Deep Image Prior and Learned Regularization Dave Van Veen A. Jalal Mahdi Soltanolkotabi Eric Price S. Vishwanath A. Dimakis 24 178 0 17 Jun 2018