Towards moderate overparameterization: global convergence guarantees for training shallow neural networks

12 February 2019

Papers citing "Towards moderate overparameterization: global convergence guarantees for training shallow neural networks"

50 / 131 papers shown

Title
Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networks Shuai Zhang Meng Wang Sijia Liu Pin-Yu Chen Jinjun Xiong UQCV MLT 71 13 0 12 Oct 2021
A global convergence theory for deep ReLU implicit networks via over-parameterization Tianxiang Gao Hailiang Liu Jia Liu Hridesh Rajan Hongyang Gao MLT 102 16 0 11 Oct 2021
Does Preprocessing Help Training Over-parameterized Neural Networks? Zhao Song Shuo Yang Ruizhe Zhang 98 50 0 09 Oct 2021
Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks Zhichao Wang Yizhe Zhu 92 20 0 20 Sep 2021
How much pre-training is enough to discover a good subnetwork? Cameron R. Wolfe Fangshuo Liao Qihan Wang Junhyung Lyle Kim Anastasios Kyrillidis 84 3 0 31 Jul 2021
Stability & Generalisation of Gradient Descent for Shallow Neural Networks without the Neural Tangent Kernel Dominic Richards Ilja Kuzborskij 69 29 0 27 Jul 2021
Nonparametric Regression with Shallow Overparameterized Neural Networks Trained by GD with Early Stopping Ilja Kuzborskij Csaba Szepesvári 98 7 0 12 Jul 2021
Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction Dominik Stöger Mahdi Soltanolkotabi ODL 80 78 0 28 Jun 2021
Early-stopped neural networks are consistent Ziwei Ji Justin D. Li Matus Telgarsky 82 37 0 10 Jun 2021
Deep Kronecker neural networks: A general framework for neural networks with adaptive activation functions Ameya Dilip Jagtap Yeonjong Shin Kenji Kawaguchi George Karniadakis ODL 111 137 0 20 May 2021
The Dynamics of Gradient Descent for Overparametrized Neural Networks Siddhartha Satpathi R. Srikant MLT AI4CE 52 14 0 13 May 2021
FL-NTK: A Neural Tangent Kernel-based Framework for Federated Learning Convergence Analysis Baihe Huang Xiaoxiao Li Zhao Song Xin Yang FedML 69 16 0 11 May 2021
Understanding Overparameterization in Generative Adversarial Networks Yogesh Balaji M. Sajedi Neha Kalibhat Mucong Ding Dominik Stöger Mahdi Soltanolkotabi Soheil Feizi AI4CE 92 21 0 12 Apr 2021
Sample Complexity and Overparameterization Bounds for Temporal Difference Learning with Neural Network Approximation Semih Cayci Siddhartha Satpathi Niao He F. I. R. Srikant 87 9 0 02 Mar 2021
Learning with invariances in random features and kernel models Song Mei Theodor Misiakiewicz Andrea Montanari OOD 107 91 0 25 Feb 2021
An Adaptive Stochastic Sequential Quadratic Programming with Differentiable Exact Augmented Lagrangians Sen Na M. Anitescu Mladen Kolar 85 44 0 10 Feb 2021
A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Network Mo Zhou Rong Ge Chi Jin 145 46 0 04 Feb 2021
Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training Cong Fang Hangfeng He Qi Long Weijie J. Su FAtt 201 172 0 29 Jan 2021
Generalization error of random features and kernel methods: hypercontractivity and kernel matrix concentration Song Mei Theodor Misiakiewicz Andrea Montanari 94 113 0 26 Jan 2021
On the Proof of Global Convergence of Gradient Descent for Deep ReLU Networks with Linear Widths Quynh N. Nguyen 126 49 0 24 Jan 2021
Non-Convex Compressed Sensing with Training Data G. Welper 60 1 0 20 Jan 2021
A Convergence Theory Towards Practical Over-parameterized Deep Neural Networks Asaf Noy Yi Tian Xu Y. Aflalo Lihi Zelnik-Manor Rong Jin 68 3 0 12 Jan 2021
Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networks Quynh N. Nguyen Marco Mondelli Guido Montúfar 87 83 0 21 Dec 2020
Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning Zeyuan Allen-Zhu Yuanzhi Li FedML 185 376 0 17 Dec 2020
Provable Benefits of Overparameterization in Model Compression: From Double Descent to Pruning Neural Networks Xiangyu Chang Yingcong Li Samet Oymak Christos Thrampoulidis 86 51 0 16 Dec 2020
Effect of the initial configuration of weights on the training and function of artificial neural networks Ricardo J. Jesus Mário Antunes R. A. D. Costa S. Dorogovtsev J. F. F. Mendes R. Aguiar 61 15 0 04 Dec 2020
Neural Contextual Bandits with Deep Representation and Shallow Exploration Pan Xu Zheng Wen Handong Zhao Quanquan Gu OffRL 89 78 0 03 Dec 2020
Coresets for Robust Training of Neural Networks against Noisy Labels Baharan Mirzasoleiman Kaidi Cao J. Leskovec NoLa 79 32 0 15 Nov 2020
Global optimality of softmax policy gradient with single hidden layer neural networks in the mean-field regime Andrea Agazzi Jianfeng Lu 84 16 0 22 Oct 2020
Beyond Lazy Training for Over-parameterized Tensor Decomposition Xiang Wang Chenwei Wu Jason D. Lee Tengyu Ma Rong Ge 83 14 0 22 Oct 2020
A Unifying View on Implicit Bias in Training Linear Neural Networks Chulhee Yun Shankar Krishnan H. Mobahi MLT 125 82 0 06 Oct 2020
It's Hard for Neural Networks To Learn the Game of Life Jacob Mitchell Springer Garrett Kenyon 85 21 0 03 Sep 2020
The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Training Andrea Montanari Yiqiao Zhong 187 97 0 25 Jul 2020
Early Stopping in Deep Networks: Double Descent and How to Eliminate it Reinhard Heckel Fatih Yilmaz 80 45 0 20 Jul 2020
Learning Over-Parametrized Two-Layer ReLU Neural Networks beyond NTK Yuanzhi Li Tengyu Ma Hongyang R. Zhang MLT 90 27 0 09 Jul 2020
When Do Neural Networks Outperform Kernel Methods? Behrooz Ghorbani Song Mei Theodor Misiakiewicz Andrea Montanari 117 189 0 24 Jun 2020
Training (Overparametrized) Neural Networks in Near-Linear Time Jan van den Brand Binghui Peng Zhao Song Omri Weinstein ODL 91 83 0 20 Jun 2020
Shape Matters: Understanding the Implicit Bias of the Noise Covariance Jeff Z. HaoChen Colin Wei Jason D. Lee Tengyu Ma 210 95 0 15 Jun 2020
Global Convergence of Sobolev Training for Overparameterized Neural Networks Jorio Cocola Paul Hand 18 6 0 14 Jun 2020
Network size and weights size for memorization with two-layers neural networks Sébastien Bubeck Ronen Eldan Y. Lee Dan Mikulincer 75 33 0 04 Jun 2020
Compressive sensing with un-trained neural networks: Gradient descent finds the smoothest approximation Reinhard Heckel Mahdi Soltanolkotabi 75 81 0 07 May 2020
A Mean-field Analysis of Deep ResNet and Beyond: Towards Provable Optimization Via Overparameterization From Depth Yiping Lu Chao Ma Yulong Lu Jianfeng Lu Lexing Ying MLT 160 79 0 11 Mar 2020
Loss landscapes and optimization in over-parameterized non-linear systems and neural networks Chaoyue Liu Libin Zhu M. Belkin ODL 98 266 0 29 Feb 2020
Global Convergence of Deep Networks with One Wide Layer Followed by Pyramidal Topology Quynh N. Nguyen Marco Mondelli ODL AI4CE 80 70 0 18 Feb 2020
Learning Parities with Neural Networks Amit Daniely Eran Malach 98 78 0 18 Feb 2020
Convergence of End-to-End Training in Deep Unsupervised Contrastive Learning Zixin Wen SSL 68 3 0 17 Feb 2020
Over-parameterized Adversarial Training: An Analysis Overcoming the Curse of Dimensionality Yi Zhang Orestis Plevrakis S. Du Xingguo Li Zhao Song Sanjeev Arora 127 53 0 16 Feb 2020
A Deep Conditioning Treatment of Neural Networks Naman Agarwal Pranjal Awasthi Satyen Kale AI4CE 107 16 0 04 Feb 2020
Memory capacity of neural networks with threshold and ReLU activations Roman Vershynin 71 21 0 20 Jan 2020
Revisiting Landscape Analysis in Deep Neural Networks: Eliminating Decreasing Paths to Infinity Shiyu Liang Ruoyu Sun R. Srikant 81 20 0 31 Dec 2019