v1v2v3 (latest)

Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation

13 February 2019

Papers citing "Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation"

50 / 211 papers shown

Title
Scaling Neural Tangent Kernels via Sketching and Random FeaturesNeural Information Processing Systems (NeurIPS), 2021 A. Zandieh Insu Han H. Avron N. Shoham Chaewon Kim Jinwoo Shin 215 35 0 15 Jun 2021
The Future is Log-Gaussian: ResNets and Their Infinite-Depth-and-Width Limit at InitializationNeural Information Processing Systems (NeurIPS), 2021 Mufan Li Mihai Nica Daniel M. Roy 295 36 0 07 Jun 2021
Symmetry-via-Duality: Invariant Neural Network Densities from Parameter-Space Correlators Anindita Maiti Keegan Stoner James Halverson 142 26 0 01 Jun 2021
Asymptotics of representation learning in finite Bayesian neural networksNeural Information Processing Systems (NeurIPS), 2021 Jacob A. Zavatone-Veth Abdulkadir Canatar Benjamin S. Ruben Cengiz Pehlevan 357 36 0 01 Jun 2021
Properties of the After Kernel Philip M. Long 164 30 0 21 May 2021
Priors in Bayesian Deep Learning: A ReviewInternational Statistical Review (ISR), 2021 Vincent Fortuin UQCV BDL 398 157 0 14 May 2021
Tensor Programs IIb: Architectural Universality of Neural Tangent Kernel Training DynamicsInternational Conference on Machine Learning (ICML), 2021 Greg Yang Etai Littwin 145 75 0 08 May 2021
Generalization Guarantees for Neural Architecture Search with Train-Validation SplitInternational Conference on Machine Learning (ICML), 2021 Samet Oymak Mingchen Li Mahdi Soltanolkotabi AI4CE OOD 238 18 0 29 Apr 2021
Learning with Neural Tangent Kernels in Near Input Sparsity Time A. Zandieh 218 0 0 01 Apr 2021
A Temporal Kernel Approach for Deep Learning with Continuous-time InformationInternational Conference on Learning Representations (ICLR), 2021 Da Xu Chuanwei Ruan Evren Körpeoglu Sushant Kumar Kannan Achan SyDa AI4TS 114 5 0 28 Mar 2021
Asymptotic Freeness of Layerwise Jacobians Caused by Invariance of Multilayer Perceptron: The Haar Orthogonal CaseCommunications in Mathematical Physics (Commun. Math. Phys.), 2021 B. Collins Tomohiro Hayase 201 8 0 24 Mar 2021
Weighted Neural Tangent Kernel: A Generalized and Improved Network-Induced KernelMachine-mediated learning (ML), 2021 Lei Tan Shutong Wu Xiaolin Huang 138 3 0 22 Mar 2021
Towards Deepening Graph Neural Networks: A GNTK-based Optimization PerspectiveInternational Conference on Learning Representations (ICLR), 2021 Wei Huang Yayong Li Weitao Du Jie Yin R. Xu Ling-Hao Chen Miao Zhang 163 19 0 03 Mar 2021
Fast Adaptation with Linearized Neural NetworksInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2021 Wesley J. Maddox Shuai Tang Pablo G. Moreno A. Wilson Andreas C. Damianou 222 34 0 02 Mar 2021
Experiments with Rich Regime Training for Deep Learning Xinyan Li A. Banerjee 153 2 0 26 Feb 2021
Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired PerspectiveInternational Conference on Learning Representations (ICLR), 2021 Wuyang Chen Xinyu Gong Zinan Lin OOD 466 269 0 23 Feb 2021
Approximation and Learning with Deep Convolutional Models: a Kernel PerspectiveInternational Conference on Learning Representations (ICLR), 2021 A. Bietti 214 32 0 19 Feb 2021
Non-asymptotic approximations of neural networks by Gaussian processesAnnual Conference Computational Learning Theory (COLT), 2021 Ronen Eldan Dan Mikulincer T. Schramm 235 24 0 17 Feb 2021
Explaining Neural Scaling LawsProceedings of the National Academy of Sciences of the United States of America (PNAS), 2021 Yasaman Bahri Ethan Dyer Jared Kaplan Jaehoon Lee Utkarsh Sharma 305 368 0 12 Feb 2021
Implicit Bias of Linear RNNsInternational Conference on Machine Learning (ICML), 2021 M Motavali Emami Mojtaba Sahraee-Ardakan Parthe Pandit S. Rangan A. Fletcher 141 13 0 19 Jan 2021
Infinitely Wide Tensor Networks as Gaussian Process Erdong Guo D. Draper 142 2 0 07 Jan 2021
Perspective: A Phase Diagram for Deep Learning unifying Jamming, Feature Learning and Lazy Training Mario Geiger Leonardo Petrini Matthieu Wyart DRL 157 11 0 30 Dec 2020
Mathematical Models of Overparameterized Neural NetworksProceedings of the IEEE (Proc. IEEE), 2020 Cong Fang Hanze Dong Tong Zhang 253 25 0 27 Dec 2020
Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep LearningInternational Conference on Learning Representations (ICLR), 2020 Zeyuan Allen-Zhu Yuanzhi Li FedML 585 433 0 17 Dec 2020
Enhanced Recurrent Neural Tangent Kernels for Non-Time-Series Data Sina Alemohammad Randall Balestriero Zichao Wang Richard Baraniuk AI4TS 120 1 0 09 Dec 2020
Generalization bounds for deep learning Guillermo Valle Pérez A. Louis BDL 239 48 0 07 Dec 2020
Feature Learning in Infinite-Width Neural Networks Greg Yang J. E. Hu MLT 384 180 0 30 Nov 2020
Implicit bias of deep linear networks in the large learning rate phase Wei Huang Weitao Du R. Xu Chunrui Liu 143 2 0 25 Nov 2020
Towards NNGP-guided Neural Architecture Search Daniel S. Park Jaehoon Lee Daiyi Peng Yuan Cao Jascha Narain Sohl-Dickstein BDL 147 34 0 11 Nov 2020
Kernel Dependence Network Chieh-Tsai Wu A. Masoomi Arthur Gretton Jennifer Dy 168 0 0 04 Nov 2020
Stable ResNetInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2020 Soufiane Hayou Eugenio Clerico Bo He George Deligiannidis Arnaud Doucet Judith Rousseau ODL SSeg 129 60 0 24 Oct 2020
Label-Aware Neural Tangent Kernel: Toward Better Generalization and Local Elasticity Shuxiao Chen Hangfeng He Weijie J. Su 121 25 0 22 Oct 2020
MimicNorm: Weight Mean and Last BN Layer Mimic the Dynamic of Batch Normalization Wen Fei Wenrui Dai Chenglin Li Junni Zou H. Xiong 180 1 0 19 Oct 2020
A Modular Analysis of Provable Acceleration via Polyak's Momentum: Training a Wide ReLU Network and a Deep Linear NetworkInternational Conference on Machine Learning (ICML), 2020 Jun-Kun Wang Chi-Heng Lin Jacob D. Abernethy 521 24 0 04 Oct 2020
Understanding Approximate Fisher Information for Fast Convergence of Natural Gradient Descent in Wide Neural NetworksNeural Information Processing Systems (NeurIPS), 2020 Ryo Karakida Kazuki Osawa 247 31 0 02 Oct 2020
Tensor Programs III: Neural Matrix Laws Greg Yang 277 53 0 22 Sep 2020
Kernel-Based Smoothness Analysis of Residual NetworksMathematical and Scientific Machine Learning (MSML), 2020 Tom Tirer Joan Bruna Raja Giryes 206 22 0 21 Sep 2020
Asymptotics of Wide Convolutional Neural Networks Anders Andreassen Ethan Dyer 199 24 0 19 Aug 2020
Neural Networks and Quantum Field Theory James Halverson Anindita Maiti Keegan Stoner 313 88 0 19 Aug 2020
Finite Versus Infinite Neural Networks: an Empirical StudyNeural Information Processing Systems (NeurIPS), 2020 Jaehoon Lee S. Schoenholz Jeffrey Pennington Ben Adlam Lechao Xiao Roman Novak Jascha Narain Sohl-Dickstein 232 227 0 31 Jul 2020
When and why PINNs fail to train: A neural tangent kernel perspectiveJournal of Computational Physics (JCP), 2020 Sizhuang He Xinling Yu P. Perdikaris 333 1,180 0 28 Jul 2020
Bayesian Deep Ensembles via the Neural Tangent KernelNeural Information Processing Systems (NeurIPS), 2020 Bobby He Balaji Lakshminarayanan Yee Whye Teh BDL UQCV 257 124 0 11 Jul 2020
Learning Over-Parametrized Two-Layer ReLU Neural Networks beyond NTKAnnual Conference Computational Learning Theory (COLT), 2020 Yuanzhi Li Tengyu Ma Hongyang R. Zhang MLT 191 28 0 09 Jul 2020
Towards an Understanding of Residual Networks Using Neural Tangent Hierarchy (NTH)CSIAM Transactions on Applied Mathematics (CSIAM Trans. Appl. Math.), 2020 Yuqing Li Yaoyu Zhang N. Yip 201 5 0 07 Jul 2020
Is SGD a Bayesian sampler? Well, almost Chris Mingard Guillermo Valle Pérez Joar Skalse A. Louis BDL 228 62 0 26 Jun 2020
The Surprising Simplicity of the Early-Time Learning Dynamics of Neural Networks Wei Hu Lechao Xiao Ben Adlam Jeffrey Pennington 162 69 0 25 Jun 2020
Tensor Programs II: Neural Tangent Kernel for Any Architecture Greg Yang 414 155 0 25 Jun 2020
On Lyapunov Exponents for RNNs: Understanding Information Propagation Using Dynamical Systems ToolsFrontiers in Applied Mathematics and Statistics (FAMS), 2020 Ryan H. Vogt M. P. Touzel Eli Shlizerman Guillaume Lajoie 205 49 0 25 Jun 2020
On the Empirical Neural Tangent Kernel of Standard Finite-Width Convolutional Neural Network Architectures M. Samarin Volker Roth David Belius 100 4 0 24 Jun 2020
Exact posterior distributions of wide Bayesian neural networks Jiri Hron Yasaman Bahri Roman Novak Jeffrey Pennington Jascha Narain Sohl-Dickstein UQCV BDL 196 30 0 18 Jun 2020