Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks

24 January 2019

Papers citing "Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks"

42 / 192 papers shown

Title
Optimization and Generalization Analysis of Transduction through Gradient Boosting and Application to Multi-scale Graph Neural Networks Kenta Oono Taiji Suzuki AI4CE 34 31 0 15 Jun 2020
Global Attention Improves Graph Networks Generalization Omri Puny Heli Ben-Hamu Y. Lipman 27 22 0 14 Jun 2020
Non-convergence of stochastic gradient descent in the training of deep neural networks Patrick Cheridito Arnulf Jentzen Florian Rossmannek 14 37 0 12 Jun 2020
Can Temporal-Difference and Q-Learning Learn Representation? A Mean-Field Theory Yufeng Zhang Qi Cai Zhuoran Yang Yongxin Chen Zhaoran Wang OOD MLT 58 11 0 08 Jun 2020
Speedy Performance Estimation for Neural Architecture Search Binxin Ru Clare Lyle Lisa Schut M. Fil Mark van der Wilk Y. Gal 13 35 0 08 Jun 2020
Feature Purification: How Adversarial Training Performs Robust Deep Learning Zeyuan Allen-Zhu Yuanzhi Li MLT AAML 27 146 0 20 May 2020
Learning the gravitational force law and other analytic functions Atish Agarwala Abhimanyu Das Rina Panigrahy Qiuyi Zhang MLT 6 0 0 15 May 2020
Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond Fanghui Liu Xiaolin Huang Yudong Chen Johan A. K. Suykens BDL 32 172 0 23 Apr 2020
Analysis of Knowledge Transfer in Kernel Regime Arman Rahbar Ashkan Panahi Chiranjib Bhattacharyya Devdatt Dubhashi M. Chehreghani 13 3 0 30 Mar 2020
Frequency Bias in Neural Networks for Input of Non-Uniform Density Ronen Basri Meirav Galun Amnon Geifman David Jacobs Yoni Kasten S. Kritchman 34 182 0 10 Mar 2020
An Optimization and Generalization Analysis for Max-Pooling Networks Alon Brutzkus Amir Globerson MLT AI4CE 11 4 0 22 Feb 2020
Learning Parities with Neural Networks Amit Daniely Eran Malach 15 76 0 18 Feb 2020
Convergence of End-to-End Training in Deep Unsupervised Contrastive Learning Zixin Wen SSL 18 2 0 17 Feb 2020
Over-parameterized Adversarial Training: An Analysis Overcoming the Curse of Dimensionality Yi Zhang Orestis Plevrakis S. Du Xingguo Li Zhao-quan Song Sanjeev Arora 21 51 0 16 Feb 2020
Proving the Lottery Ticket Hypothesis: Pruning is All You Need Eran Malach Gilad Yehudai Shai Shalev-Shwartz Ohad Shamir 48 271 0 03 Feb 2020
Memory capacity of neural networks with threshold and ReLU activations Roman Vershynin 23 21 0 20 Jan 2020
Deep Network Approximation for Smooth Functions Jianfeng Lu Zuowei Shen Haizhao Yang Shijun Zhang 35 247 0 09 Jan 2020
Towards Understanding the Spectral Bias of Deep Learning Yuan Cao Zhiying Fang Yue Wu Ding-Xuan Zhou Quanquan Gu 21 214 0 03 Dec 2019
Denoising and Regularization via Exploiting the Structural Bias of Convolutional Generators Reinhard Heckel Mahdi Soltanolkotabi DiffM 27 81 0 31 Oct 2019
Neural Spectrum Alignment: Empirical Study Dmitry Kopitkov Vadim Indelman 14 14 0 19 Oct 2019
Harnessing the Power of Infinitely Wide Deep Nets on Small-data Tasks Sanjeev Arora S. Du Zhiyuan Li Ruslan Salakhutdinov Ruosong Wang Dingli Yu AAML 9 161 0 03 Oct 2019
Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networks Yu Bai J. Lee 11 116 0 03 Oct 2019
Towards Understanding the Transferability of Deep Representations Hong Liu Mingsheng Long Jianmin Wang Michael I. Jordan 16 25 0 26 Sep 2019
Asymptotics of Wide Networks from Feynman Diagrams Ethan Dyer Guy Gur-Ari 19 113 0 25 Sep 2019
The generalization error of random features regression: Precise asymptotics and double descent curve Song Mei Andrea Montanari 39 624 0 14 Aug 2019
Kernel and Rich Regimes in Overparametrized Models Blake E. Woodworth Suriya Gunasekar Pedro H. P. Savarese E. Moroshko Itay Golan J. Lee Daniel Soudry Nathan Srebro 19 353 0 13 Jun 2019
Norm-based generalisation bounds for multi-class convolutional neural networks Antoine Ledent Waleed Mustafa Yunwen Lei Marius Kloft 8 5 0 29 May 2019
Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems Tianle Cai Ruiqi Gao Jikai Hou Siyu Chen Dong Wang Di He Zhihua Zhang Liwei Wang ODL 11 57 0 28 May 2019
What Can ResNet Learn Efficiently, Going Beyond Kernels? Zeyuan Allen-Zhu Yuanzhi Li 24 183 0 24 May 2019
Gradient Descent can Learn Less Over-parameterized Two-layer Neural Networks on Classification Problems Atsushi Nitanda Geoffrey Chinot Taiji Suzuki MLT 8 33 0 23 May 2019
A type of generalization error induced by initialization in deep neural networks Yaoyu Zhang Zhi-Qin John Xu Tao Luo Zheng Ma 9 49 0 19 May 2019
Linearized two-layers neural networks in high dimension Behrooz Ghorbani Song Mei Theodor Misiakiewicz Andrea Montanari MLT 11 241 0 27 Apr 2019
On Exact Computation with an Infinitely Wide Neural Net Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruslan Salakhutdinov Ruosong Wang 22 899 0 26 Apr 2019
A Selective Overview of Deep Learning Jianqing Fan Cong Ma Yiqiao Zhong BDL VLM 25 136 0 10 Apr 2019
Analysis of the Gradient Descent Algorithm for a Deep Neural Network Model with Skip-connections E. Weinan Chao Ma Qingcan Wang Lei Wu MLT 21 22 0 10 Apr 2019
Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks Mingchen Li Mahdi Soltanolkotabi Samet Oymak NoLa 26 350 0 27 Mar 2019
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du J. Lee Haochuan Li Liwei Wang M. Tomizuka ODL 15 1,120 0 09 Nov 2018
A Priori Estimates of the Population Risk for Two-layer Neural Networks Weinan E Chao Ma Lei Wu 24 130 0 15 Oct 2018
Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel Colin Wei J. Lee Qiang Liu Tengyu Ma 18 243 0 12 Oct 2018
PAC-Bayesian Margin Bounds for Convolutional Neural Networks Konstantinos Pitas Mike Davies P. Vandergheynst BDL 41 12 0 30 Dec 2017
High-dimensional dynamics of generalization error in neural networks Madhu S. Advani Andrew M. Saxe AI4CE 33 463 0 10 Oct 2017
Norm-Based Capacity Control in Neural Networks Behnam Neyshabur Ryota Tomioka Nathan Srebro 116 577 0 27 Feb 2015