v1v2v3 (latest)

Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation

13 February 2019

Papers citing "Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation"

50 / 211 papers shown

Title
Width and Depth Limits Commute in Residual NetworksInternational Conference on Machine Learning (ICML), 2023 Soufiane Hayou Greg Yang 166 16 0 01 Feb 2023
How does training shape the Riemannian geometry of neural network representations? Jacob A. Zavatone-Veth Sheng Yang Julian Rubinfien Cengiz Pehlevan MLT AI4CE 359 6 0 26 Jan 2023
Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn features Adityanarayanan Radhakrishnan Daniel Beaglehole Parthe Pandit M. Belkin FAtt MLT 222 16 0 28 Dec 2022
Analysis of Convolutions, Non-linearity and Depth in Graph Neural Networks using Neural Tangent Kernel Mahalakshmi Sabanayagam Pascal Esser Debarghya Ghoshdastidar 297 3 0 18 Oct 2022
Component-Wise Natural Gradient Descent -- An Efficient Neural Network OptimizationInternational Symposium on Computing and Networking - Across Practical Development and Theoretical Research (ISAPDTR), 2022 Tran van Sang Mhd Irvan R. Yamaguchi Toshiyuki Nakata 178 1 0 11 Oct 2022
Joint Embedding Self-Supervised Learning in the Kernel Regime B. Kiani Randall Balestriero Yubei Chen S. Lloyd Yann LeCun SSL 250 15 0 29 Sep 2022
Fast Neural Kernel Embeddings for General ActivationsNeural Information Processing Systems (NeurIPS), 2022 Insu Han A. Zandieh Jaehoon Lee Roman Novak Lechao Xiao Amin Karbasi 209 21 0 09 Sep 2022
Gaussian Process Surrogate Models for Neural NetworksConference on Uncertainty in Artificial Intelligence (UAI), 2022 Michael Y. Li Erin Grant Thomas Griffiths BDL SyDa 232 9 0 11 Aug 2022
Synergy and Symmetry in Deep Learning: Interactions between the Data, Model, and Inference AlgorithmInternational Conference on Machine Learning (ICML), 2022 Lechao Xiao Jeffrey Pennington 171 11 0 11 Jul 2022
Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror DescentNeural Information Processing Systems (NeurIPS), 2022 Zhiyuan Li Tianhao Wang Jason D. Lee Sanjeev Arora 253 33 0 08 Jul 2022
Informed Learning by Wide Neural Networks: Convergence, Generalization and Sampling ComplexityInternational Conference on Machine Learning (ICML), 2022 Jianyi Yang Shaolei Ren 168 3 0 02 Jul 2022
Fast Finite Width Neural Tangent KernelInternational Conference on Machine Learning (ICML), 2022 Roman Novak Jascha Narain Sohl-Dickstein S. Schoenholz AAML 176 71 0 17 Jun 2022
Large-width asymptotics for ReLU neural networks with $α$ -Stable initializations Stefano Favaro S. Fortini Stefano Peluchetti 171 2 0 16 Jun 2022
Wide Bayesian neural networks have a simple weight posterior: theory and accelerated samplingInternational Conference on Machine Learning (ICML), 2022 Jiri Hron Roman Novak Jeffrey Pennington Jascha Narain Sohl-Dickstein UQCV BDL 138 9 0 15 Jun 2022
Understanding the Generalization Benefit of Normalization Layers: Sharpness ReductionNeural Information Processing Systems (NeurIPS), 2022 Kaifeng Lyu Zhiyuan Li Sanjeev Arora FAtt 267 86 0 14 Jun 2022
Gradient Boosting Performs Gaussian Process InferenceInternational Conference on Learning Representations (ICLR), 2022 Aleksei Ustimenko Artem Beliakov Liudmila Prokhorenkova BDL 185 6 0 11 Jun 2022
Neural Collapse: A Review on Modelling Principles and Generalization Vignesh Kothapalli 355 101 0 08 Jun 2022
The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at InitializationNeural Information Processing Systems (NeurIPS), 2022 Mufan Li Mihai Nica Daniel M. Roy 279 43 0 06 Jun 2022
Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural NetworksNeural Information Processing Systems (NeurIPS), 2022 Blake Bordelon Cengiz Pehlevan MLT 303 108 0 19 May 2022
Single-level Adversarial Data Synthesis based on Neural Tangent Kernels Yu-Rong Zhang Ruei-Yang Su Sheng-Yen Chou Shan Wu GAN 393 2 0 08 Apr 2022
On the Neural Tangent Kernel Analysis of Randomly Pruned Neural NetworksInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2022 Hongru Yang Zinan Lin MLT 315 9 0 27 Mar 2022
On Connecting Deep Trigonometric Networks with Deep Gaussian Processes: Covariance, Expressivity, and Neural Tangent Kernel Chi-Ken Lu Patrick Shafto BDL 256 1 0 14 Mar 2022
More Than a Toy: Random Matrix Models Predict How Real-World Neural Representations GeneralizeInternational Conference on Machine Learning (ICML), 2022 Alexander Wei Wei Hu Jacob Steinhardt 210 85 0 11 Mar 2022
Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer Greg Yang J. E. Hu Igor Babuschkin Szymon Sidor Xiaodong Liu David Farhi Nick Ryder J. Pachocki Weizhu Chen Jianfeng Gao 293 214 0 07 Mar 2022
Contrasting random and learned features in deep Bayesian linear regressionPhysical Review E (Phys. Rev. E), 2022 Jacob A. Zavatone-Veth William L. Tong Cengiz Pehlevan BDL MLT 311 31 0 01 Mar 2022
A duality connecting neural network and cosmological dynamics Sven Krippendorf M. Spannowsky AI4CE 131 11 0 22 Feb 2022
Deep Learning in Random Neural Fields: Numerical Experiments via Neural Tangent KernelNeural Networks (NN), 2022 Kaito Watanabe Kotaro Sakamoto Ryo Karakida Sho Sonoda S. Amari OOD 188 1 0 10 Feb 2022
Deep Layer-wise Networks Have Closed-Form WeightsInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2020 Chieh-Tsai Wu A. Masoomi Arthur Gretton Jennifer Dy 293 4 0 01 Feb 2022
Neural Tangent Kernel Beyond the Infinite-Width Limit: Effects of Depth and InitializationInternational Conference on Machine Learning (ICML), 2022 Mariia Seleznova Gitta Kutyniok 388 27 0 01 Feb 2022
Kernel Methods and Multi-layer Perceptrons Learn Linear Models in High Dimensions Mojtaba Sahraee-Ardakan M. Emami Parthe Pandit S. Rangan A. Fletcher 179 9 0 20 Jan 2022
Separation of Scales and a Thermodynamic Description of Feature Learning in Some CNNsNature Communications (Nat Commun), 2021 Inbar Seroussi Gadi Naveh Zohar Ringel 281 64 0 31 Dec 2021
Eigenspace Restructuring: a Principle of Space and Frequency in Neural Networks Lechao Xiao 230 23 0 10 Dec 2021
Learning Curves for Continual Learning in Neural Networks: Self-Knowledge Transfer and Forgetting Ryo Karakida S. Akaho CLL 200 16 0 03 Dec 2021
Infinite Neural Network Quantum States: Entanglement and Training Dynamics Di Luo James Halverson 210 9 0 01 Dec 2021
Critical Initialization of Wide and Deep Neural Networks through Partial Jacobians: General Theory and Applications Darshil Doshi Tianyu He Andrey Gromov 319 10 0 23 Nov 2021
Depth induces scale-averaging in overparameterized linear Bayesian neural networks Jacob A. Zavatone-Veth Cengiz Pehlevan BDL UQCV MDE 240 12 0 23 Nov 2021
Rethinking Neural vs. Matrix-Factorization Collaborative Filtering: the Theoretical PerspectivesInternational Conference on Machine Learning (ICML), 2021 Zida Cheng Chuanwei Ruan Siheng Chen Sushant Kumar Ya Zhang 139 18 0 23 Oct 2021
Quantifying Epistemic Uncertainty in Deep Learning Ziyi Huang Henry Lam Haofeng Zhang UQCV BDL UD PER 369 15 0 23 Oct 2021
Feature Learning and Signal Propagation in Deep Neural NetworksInternational Conference on Machine Learning (ICML), 2021 Yizhang Lou Chris Mingard Yoonsoo Nam Soufiane Hayou MDE 177 18 0 22 Oct 2021
What Happens after SGD Reaches Zero Loss? --A Mathematical Framework Zhiyuan Li Tianhao Wang Sanjeev Arora MLT 313 114 0 13 Oct 2021
Imitating Deep Learning Dynamics via Locally Elastic Stochastic Differential EquationsNeural Information Processing Systems (NeurIPS), 2021 Jiayao Zhang Hua Wang Weijie J. Su 187 9 0 11 Oct 2021
On the Impact of Stable Ranks in Deep Nets B. Georgiev L. Franken Mayukh Mukherjee Georgios Arvanitidis 174 5 0 05 Oct 2021
A theory of representation learning gives a deep generalisation of kernel methodsInternational Conference on Machine Learning (ICML), 2021 Adam X. Yang Maxime Robeyns Edward Milsom Ben Anson Nandi Schoots Laurence Aitchison BDL 481 14 0 30 Aug 2021
Understanding the Generalization of Adam in Learning Neural Networks with Proper RegularizationInternational Conference on Learning Representations (ICLR), 2021 Difan Zou Yuan Cao Yuanzhi Li Quanquan Gu MLT AI4CE 283 51 0 25 Aug 2021
Nonperturbative renormalization for the neural network-QFT correspondence Harold Erbin Vincent Lahoche D. O. Samary 196 37 0 03 Aug 2021
Deep Stable neural networks: large-width asymptotics and convergence rates Stefano Favaro S. Fortini Stefano Peluchetti BDL 334 16 0 02 Aug 2021
Dataset Distillation with Infinitely Wide Convolutional NetworksNeural Information Processing Systems (NeurIPS), 2021 Timothy Nguyen Roman Novak Lechao Xiao Jaehoon Lee DD 436 273 0 27 Jul 2021
Random Neural Networks in the Infinite Width Limit as Gaussian Processes Boris Hanin BDL 198 55 0 04 Jul 2021
Convolutional Neural Bandit for Visual-aware Recommendation Yikun Ban Jingrui He 125 1 0 02 Jul 2021
Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity Arthur Jacot François Ged Berfin cSimcsek Clément Hongler Franck Gabriel 300 64 0 30 Jun 2021