A Comparative Analysis of the Optimization and Generalization Property of Two-layer Neural Network and Random Feature Models Under Gradient Descent Dynamics

8 April 2019

Chao Ma

Papers citing "A Comparative Analysis of the Optimization and Generalization Property of Two-layer Neural Network and Random Feature Models Under Gradient Descent Dynamics"

36 / 36 papers shown

Title
Nesterov acceleration in benignly non-convex landscapes Kanan Gupta Stephan Wojtowytsch 39 2 0 10 Oct 2024
Deep Learning without Global Optimization by Random Fourier Neural Networks Owen Davis Gianluca Geraci Mohammad Motamed BDL 60 0 0 16 Jul 2024
Gibbs-Based Information Criteria and the Over-Parameterized Regime Haobo Chen Yuheng Bu Greg Wornell 27 1 0 08 Jun 2023
Understanding the Initial Condensation of Convolutional Neural Networks Zhangchen Zhou Hanxu Zhou Yuqing Li Zhi-Qin John Xu MLT AI4CE 26 5 0 17 May 2023
Reinforcement Learning with Function Approximation: From Linear to Nonlinear Jihao Long Jiequn Han 27 5 0 20 Feb 2023
SPADE4: Sparsity and Delay Embedding based Forecasting of Epidemics Esha Saha L. Ho Giang Tran 36 5 0 11 Nov 2022
A Functional-Space Mean-Field Theory of Partially-Trained Three-Layer Neural Networks Zhengdao Chen Eric Vanden-Eijnden Joan Bruna MLT 27 5 0 28 Oct 2022
Empirical Phase Diagram for Three-layer Neural Networks with Infinite Width Hanxu Zhou Qixuan Zhou Zhenyuan Jin Tao Luo Yaoyu Zhang Zhi-Qin John Xu 25 20 0 24 May 2022
Beyond the Quadratic Approximation: the Multiscale Structure of Neural Network Loss Landscapes Chao Ma D. Kunin Lei Wu Lexing Ying 25 27 0 24 Apr 2022
Convergence of gradient descent for deep neural networks S. Chatterjee ODL 21 20 0 30 Mar 2022
Noise Regularizes Over-parameterized Rank One Matrix Recovery, Provably Tianyi Liu Yan Li Enlu Zhou Tuo Zhao 38 1 0 07 Feb 2022
Implicit Bias of MSE Gradient Optimization in Underparameterized Neural Networks Benjamin Bowman Guido Montúfar 28 11 0 12 Jan 2022
Convergence proof for stochastic gradient descent in the training of deep neural networks with ReLU activation for constant target functions Martin Hutzenthaler Arnulf Jentzen Katharina Pohl Adrian Riekert Luca Scarpa MLT 34 6 0 13 Dec 2021
A proof of convergence for the gradient descent optimization method with random initializations in the training of neural networks with ReLU activation for piecewise linear target functions Arnulf Jentzen Adrian Riekert 33 13 0 10 Aug 2021
Convergence analysis for gradient flows in the training of artificial neural networks with ReLU activation Arnulf Jentzen Adrian Riekert 27 23 0 09 Jul 2021
Generalization Guarantees for Neural Architecture Search with Train-Validation Split Samet Oymak Mingchen Li Mahdi Soltanolkotabi AI4CE OOD 36 13 0 29 Apr 2021
A proof of convergence for stochastic gradient descent in the training of artificial neural networks with ReLU activation for constant target functions Arnulf Jentzen Adrian Riekert MLT 32 13 0 01 Apr 2021
Convergence rates for gradient descent in the training of overparameterized artificial neural networks with biases Arnulf Jentzen T. Kröger ODL 28 7 0 23 Feb 2021
A proof of convergence for gradient descent in the training of artificial neural networks for constant target functions Patrick Cheridito Arnulf Jentzen Adrian Riekert Florian Rossmannek 28 24 0 19 Feb 2021
Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training Cong Fang Hangfeng He Qi Long Weijie J. Su FAtt 130 167 0 29 Jan 2021
On the emergence of simplex symmetry in the final and penultimate layers of neural network classifiers E. Weinan Stephan Wojtowytsch 30 43 0 10 Dec 2020
Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't E. Weinan Chao Ma Stephan Wojtowytsch Lei Wu AI4CE 22 133 0 22 Sep 2020
The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Training Andrea Montanari Yiqiao Zhong 47 95 0 25 Jul 2020
A Revision of Neural Tangent Kernel-based Approaches for Neural Networks Kyungsu Kim A. Lozano Eunho Yang AAML 27 0 0 02 Jul 2020
Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theory Tao Luo Haizhao Yang 32 73 0 28 Jun 2020
Non-convergence of stochastic gradient descent in the training of deep neural networks Patrick Cheridito Arnulf Jentzen Florian Rossmannek 14 37 0 12 Jun 2020
Can Shallow Neural Networks Beat the Curse of Dimensionality? A mean field training perspective Stephan Wojtowytsch E. Weinan MLT 26 48 0 21 May 2020
Overall error analysis for the training of deep neural networks via stochastic gradient descent with random initialisation Arnulf Jentzen Timo Welti 17 15 0 03 Mar 2020
Learning Parities with Neural Networks Amit Daniely Eran Malach 24 76 0 18 Feb 2020
Machine Learning from a Continuous Viewpoint E. Weinan Chao Ma Lei Wu 27 102 0 30 Dec 2019
The Local Elasticity of Neural Networks Hangfeng He Weijie J. Su 40 44 0 15 Oct 2019
A type of generalization error induced by initialization in deep neural networks Yaoyu Zhang Zhi-Qin John Xu Tao Luo Zheng Ma 9 49 0 19 May 2019
Analysis of the Gradient Descent Algorithm for a Deep Neural Network Model with Skip-connections E. Weinan Chao Ma Qingcan Wang Lei Wu MLT 32 22 0 10 Apr 2019
Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks Mingchen Li Mahdi Soltanolkotabi Samet Oymak NoLa 47 351 0 27 Mar 2019
A Priori Estimates of the Population Risk for Two-layer Neural Networks Weinan E Chao Ma Lei Wu 29 130 0 15 Oct 2018
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 308 2,890 0 15 Sep 2016