Global Convergence of Adaptive Gradient Methods for An Over-parameterized Neural Network

19 February 2019

Papers citing "Global Convergence of Adaptive Gradient Methods for An Over-parameterized Neural Network"

16 / 16 papers shown

Title
Regularized Gradient Clipping Provably Trains Wide and Deep Neural Networks Matteo Tucat Anirbit Mukherjee Procheta Sen Mingfei Sun Omar Rivasplata MLT 39 1 0 12 Apr 2024
Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedup Yan Sun Li Shen Hao Sun Liang Ding Dacheng Tao FedML 29 17 0 30 Jul 2023
Global Convergence of SGD On Two Layer Neural Nets Pulkit Gopalani Anirbit Mukherjee 26 5 0 20 Oct 2022
Provable Regret Bounds for Deep Online Learning and Control Xinyi Chen Edgar Minasyan Jason D. Lee Elad Hazan 36 6 0 15 Oct 2021
Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks Zhichao Wang Yizhe Zhu 37 18 0 20 Sep 2021
The Low-Rank Simplicity Bias in Deep Networks Minyoung Huh H. Mobahi Richard Y. Zhang Brian Cheung Pulkit Agrawal Phillip Isola 30 110 0 18 Mar 2021
A Convergence Theory Towards Practical Over-parameterized Deep Neural Networks Asaf Noy Yi Tian Xu Y. Aflalo Lihi Zelnik-Manor Rong Jin 41 3 0 12 Jan 2021
Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networks Quynh N. Nguyen Marco Mondelli Guido Montúfar 25 81 0 21 Dec 2020
Training (Overparametrized) Neural Networks in Near-Linear Time Jan van den Brand Binghui Peng Zhao Song Omri Weinstein ODL 29 82 0 20 Jun 2020
Provable Training of a ReLU Gate with an Iterative Non-Gradient Algorithm Sayar Karmakar Anirbit Mukherjee 14 7 0 08 May 2020
Adaptive Federated Optimization Sashank J. Reddi Zachary B. Charles Manzil Zaheer Zachary Garrett Keith Rush Jakub Konecný Sanjiv Kumar H. B. McMahan FedML 58 1,395 0 29 Feb 2020
Convergence of End-to-End Training in Deep Unsupervised Contrastive Learning Zixin Wen SSL 21 2 0 17 Feb 2020
Towards Understanding the Spectral Bias of Deep Learning Yuan Cao Zhiying Fang Yue Wu Ding-Xuan Zhou Quanquan Gu 41 215 0 03 Dec 2019
Overparameterized Neural Networks Implement Associative Memory Adityanarayanan Radhakrishnan M. Belkin Caroline Uhler BDL 35 71 0 26 Sep 2019
Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks Guodong Zhang James Martens Roger C. Grosse ODL 22 124 0 27 May 2019
Gradient Descent can Learn Less Over-parameterized Two-layer Neural Networks on Classification Problems Atsushi Nitanda Geoffrey Chinot Taiji Suzuki MLT 16 33 0 23 May 2019