v1v2v3v4 (latest)

Neural Tangent Kernel: Convergence and Generalization in Neural Networks

20 June 2018

Papers citing "Neural Tangent Kernel: Convergence and Generalization in Neural Networks"

50 / 2,408 papers shown

Title
Explicitizing an Implicit Bias of the Frequency Principle in Two-layer Neural Networks Yaoyu Zhang Zhi-Qin John Xu Yaoyu Zhang Zheng Ma MLT AI4CE 214 43 0 24 May 2019
Neural Temporal-Difference and Q-Learning Provably Converge to Global OptimaMathematics of Operations Research (MOR), 2019 Qi Cai Zhuoran Yang Jason D. Lee Zhaoran Wang 180 35 0 24 May 2019
Gradient Descent can Learn Less Over-parameterized Two-layer Neural Networks on Classification Problems Atsushi Nitanda Geoffrey Chinot Taiji Suzuki MLT 198 36 0 23 May 2019
A type of generalization error induced by initialization in deep neural networksMathematical and Scientific Machine Learning (MSML), 2019 Yaoyu Zhang Zhi-Qin John Xu Yaoyu Zhang Zheng Ma 257 52 0 19 May 2019
An Information Theoretic Interpretation to Deep Neural NetworksInternational Symposium on Information Theory (ISIT), 2019 Shao-Lun Huang Xiangxiang Xu Lizhong Zheng G. Wornell FAtt 176 46 0 16 May 2019
Do Kernel and Neural Embeddings Help in Training and Generalization?Neural Processing Letters (NPL), 2019 Arman Rahbar Emilio Jorge Devdatt Dubhashi Morteza Haghir Chehreghani MLT 172 1 0 13 May 2019
The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical StudyInternational Conference on Machine Learning (ICML), 2019 Daniel S. Park Jascha Narain Sohl-Dickstein Quoc V. Le Samuel L. Smith 149 60 0 09 May 2019
Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz AugmentationNeural Information Processing Systems (NeurIPS), 2019 Colin Wei Tengyu Ma 307 122 0 09 May 2019
Similarity of Neural Network Representations RevisitedInternational Conference on Machine Learning (ICML), 2019 Simon Kornblith Mohammad Norouzi Honglak Lee Geoffrey E. Hinton 1.2K 1,722 0 01 May 2019
Linearized two-layers neural networks in high dimensionAnnals of Statistics (Ann. Stat.), 2019 Behrooz Ghorbani Song Mei Theodor Misiakiewicz Andrea Montanari MLT 226 255 0 27 Apr 2019
On Exact Computation with an Infinitely Wide Neural Net Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruslan Salakhutdinov Ruosong Wang 577 982 0 26 Apr 2019
Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like process Guy Blanc Neha Gupta Gregory Valiant Paul Valiant 315 158 0 19 Apr 2019
The Impact of Neural Network Overparameterization on Gradient Confusion and Stochastic Gradient Descent Karthik A. Sankararaman Soham De Zheng Xu Wenjie Huang Tom Goldstein ODL 232 114 0 15 Apr 2019
A Selective Overview of Deep Learning Jianqing Fan Cong Ma Yiqiao Zhong BDL VLM 359 143 0 10 Apr 2019
Analysis of the Gradient Descent Algorithm for a Deep Neural Network Model with Skip-connections E. Weinan Chao Ma Qingcan Wang Lei Wu MLT 234 22 0 10 Apr 2019
A Comparative Analysis of the Optimization and Generalization Property of Two-layer Neural Network and Random Feature Models Under Gradient Descent Dynamics E. Weinan Chao Ma Lei Wu MLT 178 124 0 08 Apr 2019
Convergence rates for the stochastic gradient descent method for non-convex objective functions Benjamin J. Fehrman Benjamin Gess Arnulf Jentzen 282 110 0 02 Apr 2019
On the Power and Limitations of Random Features for Understanding Neural Networks Gilad Yehudai Ohad Shamir MLT 282 187 0 01 Apr 2019
Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks Mingchen Li Mahdi Soltanolkotabi Samet Oymak NoLa 394 374 0 27 Mar 2019
General Probabilistic Surface Optimization and Log Density Estimation Dmitry Kopitkov Vadim Indelman 337 1 0 25 Mar 2019
Towards Characterizing Divergence in Deep Q-Learning Joshua Achiam Ethan Knight Pieter Abbeel 163 101 0 21 Mar 2019
Surprises in High-Dimensional Ridgeless Least Squares InterpolationAnnals of Statistics (Ann. Stat.), 2019 Trevor Hastie Andrea Montanari Saharon Rosset Robert Tibshirani 815 806 0 19 Mar 2019
Stabilize Deep ResNet with A Sharp Scaling Factor $τ$ Huishuai Zhang Da Yu Mingyang Yi Wei Chen Tie-Yan Liu 331 11 0 17 Mar 2019
Mean Field Analysis of Deep Neural Networks Justin A. Sirignano K. Spiliopoulos 204 85 0 11 Mar 2019
Function Space Particle Optimization for Bayesian Neural NetworksInternational Conference on Learning Representations (ICLR), 2019 Ziyu Wang Zhaolin Ren Jun Zhu Bo Zhang BDL 165 68 0 26 Feb 2019
Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent Jaehoon Lee Lechao Xiao S. Schoenholz Yasaman Bahri Roman Novak Jascha Narain Sohl-Dickstein Jeffrey Pennington 562 1,203 0 18 Feb 2019
Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limit Song Mei Theodor Misiakiewicz Andrea Montanari MLT 275 301 0 16 Feb 2019
Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation Greg Yang 464 296 0 13 Feb 2019
Uniform convergence may be unable to explain generalization in deep learningNeural Information Processing Systems (NeurIPS), 2019 Vaishnavh Nagarajan J. Zico Kolter MoMe AI4CE 392 333 0 13 Feb 2019
Mean Field Limit of the Learning Dynamics of Multilayer Neural Networks Phan-Minh Nguyen AI4CE 155 72 0 07 Feb 2019
Are All Layers Created Equal? Chiyuan Zhang Samy Bengio Y. Singer 274 156 0 06 Feb 2019
Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Networks Yuan Cao Quanquan Gu ODL MLT AI4CE 520 166 0 04 Feb 2019
Stiffness: A New Perspective on Generalization in Neural Networks Stanislav Fort Pawel Krzysztof Nowak Stanislaw Jastrzebski S. Narayanan 244 105 0 28 Jan 2019
Dynamical Isometry and a Mean Field Theory of LSTMs and GRUs D. Gilboa B. Chang Minmin Chen Greg Yang S. Schoenholz Ed H. Chi Jeffrey Pennington 208 43 0 25 Jan 2019
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruosong Wang MLT 557 1,024 0 24 Jan 2019
Training Neural Networks as Learning Data-adaptive Kernels: Provable Representation and Approximation Benefits Xialiang Dou Tengyuan Liang MLT 228 42 0 21 Jan 2019
A Theoretical Analysis of Deep Q-Learning Jianqing Fan Zhuoran Yang Yuchen Xie Zhaoran Wang 491 692 0 01 Jan 2019
On the Benefit of Width for Neural Networks: Disappearance of Bad Basins Dawei Li Tian Ding Tian Ding 447 41 0 28 Dec 2018
On Lazy Training in Differentiable Programming Lénaïc Chizat Edouard Oyallon Francis R. Bach 466 902 0 19 Dec 2018
Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers Zeyuan Allen-Zhu Yuanzhi Li Yingyu Liang MLT 723 811 0 12 Nov 2018
A Convergence Theory for Deep Learning via Over-ParameterizationInternational Conference on Machine Learning (ICML), 2018 Zeyuan Allen-Zhu Yuanzhi Li Zhao Song AI4CE ODL 852 1,546 0 09 Nov 2018
Gradient Descent Finds Global Minima of Deep Neural NetworksInternational Conference on Machine Learning (ICML), 2018 S. Du Jason D. Lee Haochuan Li Liwei Wang Masayoshi Tomizuka ODL 763 1,185 0 09 Nov 2018
On the Convergence Rate of Training Recurrent Neural Networks Zeyuan Allen-Zhu Yuanzhi Li Zhao Song 544 198 0 29 Oct 2018
A jamming transition from under- to over-parametrization affects loss landscape and generalization S. Spigler Mario Geiger Stéphane dÁscoli Levent Sagun Giulio Biroli Matthieu Wyart 329 158 0 22 Oct 2018
Exchangeability and Kernel Invariance in Trained MLPs Russell Tsuchida Fred Roosta M. Gallagher 128 3 0 19 Oct 2018
Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel Colin Wei Jason D. Lee Qiang Liu Tengyu Ma 623 260 0 12 Oct 2018
Information Geometry of Orthogonal Initializations and Training Piotr A. Sokól Il-Su Park AI4CE 251 17 0 09 Oct 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 667 1,334 0 04 Oct 2018
Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning Charles H. Martin Michael W. Mahoney AI4CE 337 229 0 02 Oct 2018
Generalization Properties of hyper-RKHS and its ApplicationsJournal of machine learning research (JMLR), 2018 Fanghui Liu Lei Shi Xiaolin Huang Jie Yang Johan A. K. Suykens 174 7 0 26 Sep 2018