Papers citing 'A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks'

Title
GraphNorm: A Principled Approach to Accelerating Graph Neural Network TrainingInternational Conference on Machine Learning (ICML), 2020 Tianle Cai Shengjie Luo Keyulu Xu Di He Tie-Yan Liu Liwei Wang GNN 240 197 0 07 Sep 2020
Holdout SGD: Byzantine Tolerant Federated Learning Shahar Azulay Lior Raz Amir Globerson Tomer Koren Y. Afek FedML 169 7 0 11 Aug 2020
Universality of Gradient Descent Neural Network TrainingNeural Networks (NN), 2020 G. Welper 129 11 0 27 Jul 2020
On regularization of gradient descent, layer imbalance and flat minima Boris Ginsburg 36 2 0 18 Jul 2020
From Symmetry to Geometry: Tractable Nonconvex Problems Yuqian Zhang Qing Qu John N. Wright 287 48 0 14 Jul 2020
Maximum-and-Concatenation NetworksInternational Conference on Machine Learning (ICML), 2020 Xingyu Xie Hao Kong Yue Yu Wayne Zhang Guangcan Liu Zhouchen Lin 244 2 0 09 Jul 2020
Towards an Understanding of Residual Networks Using Neural Tangent Hierarchy (NTH)CSIAM Transactions on Applied Mathematics (CSIAM Trans. Appl. Math.), 2020 Yuqing Li Yaoyu Zhang N. Yip 201 5 0 07 Jul 2020
Deep Polynomial Neural Networks Grigorios G. Chrysos Stylianos Moschoglou Giorgos Bouritsas Jiankang Deng Yannis Panagakis Stefanos Zafeiriou 143 110 0 20 Jun 2020
Directional Pruning of Deep Neural Networks Shih-Kang Chao Zhanyu Wang Yue Xing Guang Cheng ODL 211 35 0 16 Jun 2020
Implicit Regularization in Deep Learning May Not Be Explainable by Norms Noam Razin Nadav Cohen 266 163 0 13 May 2020
Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement LearningJournal of machine learning research (JMLR), 2020 Tabish Rashid Mikayel Samvelyan Christian Schroeder de Witt Gregory Farquhar Jakob N. Foerster Shimon Whiteson 428 1,107 0 19 Mar 2020
On Alignment in Deep Linear Neural Networks Adityanarayanan Radhakrishnan Eshaan Nichani D. Bernstein Caroline Uhler 129 2 0 13 Mar 2020
Batch Normalization Provably Avoids Rank Collapse for Randomly Initialised Deep Networks Hadi Daneshmand Jonas Köhler Francis R. Bach Thomas Hofmann Aurelien Lucchi OOD ODL 145 4 0 03 Mar 2020
On the Global Convergence of Training Deep Linear ResNetsInternational Conference on Learning Representations (ICLR), 2020 Difan Zou Philip M. Long Quanquan Gu 170 41 0 02 Mar 2020
De-randomized PAC-Bayes Margin Bounds: Applications to Non-convex and Non-smooth Predictors A. Banerjee Tiancong Chen Yingxue Zhou BDL 213 9 0 23 Feb 2020
Revealing the Structure of Deep Neural Networks via Convex DualityInternational Conference on Machine Learning (ICML), 2020 Tolga Ergen Mert Pilanci MLT 366 74 0 22 Feb 2020
The Break-Even Point on Optimization Trajectories of Deep Neural NetworksInternational Conference on Learning Representations (ICLR), 2020 Stanislaw Jastrzebski Maciej Szymczak Stanislav Fort Devansh Arpit Jacek Tabor Dong Wang Krzysztof J. Geras 221 183 0 21 Feb 2020
Almost Sure Convergence of Dropout Algorithms for Neural Networks Albert Senen-Cerda J. Sanders 210 11 0 06 Feb 2020
A Deep Conditioning Treatment of Neural NetworksInternational Conference on Algorithmic Learning Theory (ALT), 2020 Naman Agarwal Pranjal Awasthi Satyen Kale AI4CE 292 18 0 04 Feb 2020
Provable Benefit of Orthogonal Initialization in Optimizing Deep Linear NetworksInternational Conference on Learning Representations (ICLR), 2020 Wei Hu Lechao Xiao Jeffrey Pennington 169 126 0 16 Jan 2020
Optimization for deep learning: theory and algorithms Tian Ding ODL 275 177 0 19 Dec 2019
Over-parametrized deep neural networks do not generalize well Michael Kohler A. Krzyżak 145 15 0 09 Dec 2019
Analysis of the rate of convergence of neural network regression estimates which are easy to implement Alina Braun Michael Kohler A. Krzyżak 155 1 0 09 Dec 2019
On the rate of convergence of a neural network regression estimate learned by gradient descent Alina Braun Michael Kohler Harro Walk 99 11 0 09 Dec 2019
Correction Filter for Single Image Super-Resolution: Robustifying Off-the-Shelf Deep Super-ResolversComputer Vision and Pattern Recognition (CVPR), 2019 Shady Abu Hussein Tom Tirer Raja Giryes SupR 159 74 0 30 Nov 2019
AR-Net: A simple Auto-Regressive Neural Network for time-series Oskar Triebe N. Laptev Ram Rajagopal AI4TS AI4CE 212 67 0 27 Nov 2019
Global Convergence of Gradient Descent for Deep Linear Residual NetworksNeural Information Processing Systems (NeurIPS), 2019 Lei Wu Qingcan Wang Chao Ma ODL AI4CE 223 24 0 02 Nov 2019
Neural Similarity LearningNeural Information Processing Systems (NeurIPS), 2019 Weiyang Liu Zhen Liu James M. Rehg Le Song 157 32 0 28 Oct 2019
Effects of Depth, Width, and Initialization: A Convergence Analysis of Layer-wise Training for Deep Linear Neural NetworksAnalysis and Applications (Anal. Appl.), 2019 Yeonjong Shin 259 13 0 14 Oct 2019
Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizersInformation and Inference A Journal of the IMA (JIII), 2019 B. Bah Holger Rauhut Ulrich Terstiege Michael Westdickenberg MLT 276 77 0 12 Oct 2019
Pure and Spurious Critical Points: a Geometric Study of Linear NetworksInternational Conference on Learning Representations (ICLR), 2019 Matthew Trager Kathlén Kohn Joan Bruna 148 36 0 03 Oct 2019
Student Specialization in Deep ReLU Networks With Finite Width and Input Dimension Yuandong Tian MLT 180 8 0 30 Sep 2019
Overparameterized Neural Networks Implement Associative MemoryProceedings of the National Academy of Sciences of the United States of America (PNAS), 2019 Adityanarayanan Radhakrishnan M. Belkin Caroline Uhler BDL 199 77 0 26 Sep 2019
Stochastic AUC Maximization with Deep Neural NetworksInternational Conference on Learning Representations (ICLR), 2019 Mingrui Liu Zhuoning Yuan Yiming Ying Tianbao Yang 343 113 0 28 Aug 2019
PolyGAN: High-Order Polynomial Generators Grigorios G. Chrysos Stylianos Moschoglou Yannis Panagakis Stefanos Zafeiriou GAN 130 19 0 19 Aug 2019
Effect of Activation Functions on the Training of Overparametrized Neural NetsInternational Conference on Learning Representations (ICLR), 2019 A. Panigrahi Abhishek Shetty Navin Goyal 193 23 0 16 Aug 2019
Hessian based analysis of SGD for Deep Nets: Dynamics and GeneralizationSDM (SDM), 2019 Xinyan Li Qilong Gu Yingxue Zhou Tiancong Chen A. Banerjee ODL 180 54 0 24 Jul 2019
Towards Enhancing Fault Tolerance in Neural NetworksInternational Conference on Mobile and Ubiquitous Systems: Networking and Services (MobiQuitous), 2019 Vasisht Duddu D. V. Rao V. Balas OOD AAML 155 6 0 06 Jul 2019
Learning One-hidden-layer neural networks via Provable Gradient Descent with Random Initialization Shuhao Xia Yuanming Shi ODL MLT 82 0 0 04 Jul 2019
Decoupling Gating from Linearity Jonathan Fiat Eran Malach Shai Shalev-Shwartz 184 31 0 12 Jun 2019
Implicit Regularization in Deep Matrix FactorizationNeural Information Processing Systems (NeurIPS), 2019 Sanjeev Arora Nadav Cohen Wei Hu Yuping Luo AI4CE 352 556 0 31 May 2019
On the Expressive Power of Deep Polynomial Neural NetworksNeural Information Processing Systems (NeurIPS), 2019 Joe Kileel Matthew Trager Joan Bruna 152 91 0 29 May 2019
On Dropout and Nuclear Norm RegularizationInternational Conference on Machine Learning (ICML), 2019 Poorya Mianjy R. Arora 203 24 0 28 May 2019
Fast Convergence of Natural Gradient Descent for Overparameterized Neural NetworksNeural Information Processing Systems (NeurIPS), 2019 Guodong Zhang James Martens Roger C. Grosse ODL 238 130 0 27 May 2019
The Impact of Neural Network Overparameterization on Gradient Confusion and Stochastic Gradient Descent Karthik A. Sankararaman Soham De Zheng Xu Wenjie Huang Tom Goldstein ODL 236 114 0 15 Apr 2019
Connections Between Adaptive Control and Optimization in Machine Learning Joseph E. Gaudio T. Gibson Anuradha M. Annaswamy M. Bolender E. Lavretsky AI4CE 103 44 0 11 Apr 2019
Analysis of the Gradient Descent Algorithm for a Deep Neural Network Model with Skip-connections E. Weinan Chao Ma Qingcan Wang Lei Wu MLT 234 22 0 10 Apr 2019
Every Local Minimum Value is the Global Minimum Value of Induced Model in Non-convex Machine Learning Kenji Kawaguchi Jiaoyang Huang L. Kaelbling AAML 167 19 0 07 Apr 2019
Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Networks Yuan Cao Quanquan Gu ODL MLT AI4CE 535 166 0 04 Feb 2019
Depth creates no more spurious local minima Li Zhang 189 19 0 28 Jan 2019

GraphNorm: A Principled Approach to Accelerating Graph Neural Network TrainingInternational Conference on Machine Learning (ICML), 2020

Tianle Cai

240

197

0

07 Sep 2020

Holdout SGD: Byzantine Tolerant Federated Learning

169

7

0

11 Aug 2020

Universality of Gradient Descent Neural Network TrainingNeural Networks (NN), 2020

G. Welper

129

11

0

27 Jul 2020

On regularization of gradient descent, layer imbalance and flat minima

Boris Ginsburg

36

2

0

18 Jul 2020

From Symmetry to Geometry: Tractable Nonconvex Problems

Yuqian Zhang

Qing Qu

John N. Wright

287

48

0

14 Jul 2020

Maximum-and-Concatenation NetworksInternational Conference on Machine Learning (ICML), 2020

244

2

0

09 Jul 2020

Towards an Understanding of Residual Networks Using Neural Tangent Hierarchy (NTH)CSIAM Transactions on Applied Mathematics (CSIAM Trans. Appl. Math.), 2020

Yuqing Li

Yaoyu Zhang

N. Yip

201

5

0

07 Jul 2020

Deep Polynomial Neural Networks

Yannis Panagakis

143

110

0

20 Jun 2020

Directional Pruning of Deep Neural Networks

211

35

0

16 Jun 2020

Implicit Regularization in Deep Learning May Not Be Explainable by Norms

Noam Razin

Nadav Cohen

266

163

0

13 May 2020

Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement LearningJournal of machine learning research (JMLR), 2020

Tabish Rashid

Mikayel Samvelyan

Christian Schroeder de Witt

Gregory Farquhar

Jakob N. Foerster

Shimon Whiteson

428

1,107

0

19 Mar 2020

On Alignment in Deep Linear Neural Networks

Adityanarayanan Radhakrishnan

Eshaan Nichani

D. Bernstein

Caroline Uhler

129

2

0

13 Mar 2020

Batch Normalization Provably Avoids Rank Collapse for Randomly Initialised Deep Networks

145

4

0

03 Mar 2020

On the Global Convergence of Training Deep Linear ResNetsInternational Conference on Learning Representations (ICLR), 2020

Difan Zou

Philip M. Long

Quanquan Gu

170

41

0

02 Mar 2020

De-randomized PAC-Bayes Margin Bounds: Applications to Non-convex and Non-smooth Predictors

213

9

0

23 Feb 2020

Revealing the Structure of Deep Neural Networks via Convex DualityInternational Conference on Machine Learning (ICML), 2020

Tolga Ergen

Mert Pilanci

MLT

366

74

0

22 Feb 2020

The Break-Even Point on Optimization Trajectories of Deep Neural NetworksInternational Conference on Learning Representations (ICLR), 2020

Stanislaw Jastrzebski

221

183

0

21 Feb 2020

Almost Sure Convergence of Dropout Algorithms for Neural Networks

Albert Senen-Cerda

J. Sanders

210

11

0

06 Feb 2020

A Deep Conditioning Treatment of Neural NetworksInternational Conference on Algorithmic Learning Theory (ALT), 2020

292

18

0

04 Feb 2020

Provable Benefit of Orthogonal Initialization in Optimizing Deep Linear NetworksInternational Conference on Learning Representations (ICLR), 2020

Wei Hu

Lechao Xiao

Jeffrey Pennington

169

126

0

16 Jan 2020

Optimization for deep learning: theory and algorithms

Tian Ding

ODL

275

177

0

19 Dec 2019

Over-parametrized deep neural networks do not generalize well

Michael Kohler

A. Krzyżak

145

15

0

09 Dec 2019

Analysis of the rate of convergence of neural network regression estimates which are easy to implement

Alina Braun

Michael Kohler

A. Krzyżak

155

1

0

09 Dec 2019

On the rate of convergence of a neural network regression estimate learned by gradient descent

Alina Braun

Michael Kohler

Harro Walk

99

11

0

09 Dec 2019

Correction Filter for Single Image Super-Resolution: Robustifying Off-the-Shelf Deep Super-ResolversComputer Vision and Pattern Recognition (CVPR), 2019

159

74

0

30 Nov 2019

AR-Net: A simple Auto-Regressive Neural Network for time-series

212

67

0

27 Nov 2019

Global Convergence of Gradient Descent for Deep Linear Residual NetworksNeural Information Processing Systems (NeurIPS), 2019

Lei Wu

Qingcan Wang

Chao Ma

ODL AI4CE

223

24

0

02 Nov 2019

Neural Similarity LearningNeural Information Processing Systems (NeurIPS), 2019

157

32

0

28 Oct 2019

Effects of Depth, Width, and Initialization: A Convergence Analysis of Layer-wise Training for Deep Linear Neural NetworksAnalysis and Applications (Anal. Appl.), 2019

Yeonjong Shin

259

13

0

14 Oct 2019

Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizersInformation and Inference A Journal of the IMA (JIII), 2019

B. Bah

Holger Rauhut

Ulrich Terstiege

Michael Westdickenberg

MLT

276

77

0

12 Oct 2019

Pure and Spurious Critical Points: a Geometric Study of Linear NetworksInternational Conference on Learning Representations (ICLR), 2019

Matthew Trager

Kathlén Kohn

Joan Bruna

148

36

0

03 Oct 2019

Student Specialization in Deep ReLU Networks With Finite Width and Input Dimension

Yuandong Tian

MLT

180

8

0

30 Sep 2019

Overparameterized Neural Networks Implement Associative MemoryProceedings of the National Academy of Sciences of the United States of America (PNAS), 2019

Adityanarayanan Radhakrishnan

M. Belkin

Caroline Uhler

BDL

199

77

0

26 Sep 2019

Stochastic AUC Maximization with Deep Neural NetworksInternational Conference on Learning Representations (ICLR), 2019

343

113

0

28 Aug 2019

PolyGAN: High-Order Polynomial Generators

Yannis Panagakis

130

19

0

19 Aug 2019

Effect of Activation Functions on the Training of Overparametrized Neural NetsInternational Conference on Learning Representations (ICLR), 2019

A. Panigrahi

Abhishek Shetty

Navin Goyal

193

23

0

16 Aug 2019

Hessian based analysis of SGD for Deep Nets: Dynamics and GeneralizationSDM (SDM), 2019

180

54

0

24 Jul 2019

Towards Enhancing Fault Tolerance in Neural NetworksInternational Conference on Mobile and Ubiquitous Systems: Networking and Services (MobiQuitous), 2019

155

6

0

06 Jul 2019

Learning One-hidden-layer neural networks via Provable Gradient Descent with Random Initialization

Shuhao Xia

Yuanming Shi

ODL MLT