Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networks

6 April 2023

Papers citing "Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networks"

25 / 25 papers shown

Title
Don't be lazy: CompleteP enables compute-efficient deep transformers Nolan Dey Bin Claire Zhang Lorenzo Noci Mufan Bill Li Blake Bordelon Shane Bergsma C. Pehlevan Boris Hanin Joel Hestness 37 0 0 02 May 2025
Uncertainty Quantification From Scaling Laws in Deep Neural Networks Ibrahim Elsharkawy Yonatan Kahn Benjamin Hooberman UQCV 45 0 0 07 Mar 2025
Asymptotic Analysis of Two-Layer Neural Networks after One Gradient Step under Gaussian Mixtures Data with Structure Samet Demir Zafer Dogan MLT 34 0 0 02 Mar 2025
Function-Space Learning Rates Edward Milsom Ben Anson Laurence Aitchison 41 1 0 24 Feb 2025
Universal Sharpness Dynamics in Neural Network Training: Fixed Point Analysis, Edge of Stability, and Route to Chaos Dayal Singh Kalra Tianyu He M. Barkeshli 47 4 0 17 Feb 2025
Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer Blake Bordelon C. Pehlevan AI4CE 59 1 0 04 Feb 2025
The Optimization Landscape of SGD Across the Feature Learning Strength Alexander B. Atanasov Alexandru Meterez James B. Simon C. Pehlevan 43 2 0 06 Oct 2024
Bayesian RG Flow in Neural Network Field Theories Jessica N. Howard Marc S. Klinger Anindita Maiti A. G. Stapleton 60 1 0 27 May 2024
Dissecting the Interplay of Attention Paths in a Statistical Mechanics Theory of Transformers Lorenzo Tiberi Francesca Mignacco Kazuki Irie H. Sompolinsky 39 5 0 24 May 2024
Infinite Limits of Multi-head Transformer Dynamics Blake Bordelon Hamza Tahir Chaudhry C. Pehlevan AI4CE 35 9 0 24 May 2024
Asymptotics of feature learning in two-layer networks after one gradient-step Hugo Cui Luca Pesce Yatin Dandi Florent Krzakala Yue M. Lu Lenka Zdeborová Bruno Loureiro MLT 44 16 0 07 Feb 2024
A Dynamical Model of Neural Scaling Laws Blake Bordelon Alexander B. Atanasov C. Pehlevan 44 36 0 02 Feb 2024
More is Better in Modern Machine Learning: when Infinite Overparameterization is Optimal and Overfitting is Obligatory James B. Simon Dhruva Karkada Nikhil Ghosh Mikhail Belkin AI4CE BDL 18 13 0 24 Nov 2023
Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit Blake Bordelon Lorenzo Noci Mufan Bill Li Boris Hanin C. Pehlevan 14 22 0 28 Sep 2023
Small-scale proxies for large-scale Transformer training instabilities Mitchell Wortsman Peter J. Liu Lechao Xiao Katie Everett A. Alemi ... Jascha Narain Sohl-Dickstein Kelvin Xu Jaehoon Lee Justin Gilmer Simon Kornblith 30 80 0 25 Sep 2023
Speed Limits for Deep Learning Inbar Seroussi Alexander A. Alemi M. Helias Z. Ringel 11 0 0 27 Jul 2023
Neural Network Field Theories: Non-Gaussianity, Actions, and Locality M. Demirtaş James Halverson Anindita Maiti M. Schwartz Keegan Stoner AI4CE 16 10 0 06 Jul 2023
How Two-Layer Neural Networks Learn, One (Giant) Step at a Time Yatin Dandi Florent Krzakala Bruno Loureiro Luca Pesce Ludovic Stephan MLT 27 25 0 29 May 2023
Feature-Learning Networks Are Consistent Across Widths At Realistic Scales Nikhil Vyas Alexander B. Atanasov Blake Bordelon Depen Morwani Sabarish Sainathan C. Pehlevan 17 22 0 28 May 2023
Phase diagram of early training dynamics in deep neural networks: effect of the learning rate, depth, and width Dayal Singh Kalra M. Barkeshli 11 9 0 23 Feb 2023
Meta-Principled Family of Hyperparameter Scaling Strategies Sho Yaida 50 16 0 10 Oct 2022
The Influence of Learning Rule on Representation Dynamics in Wide Neural Networks Blake Bordelon C. Pehlevan 38 22 0 05 Oct 2022
Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks Blake Bordelon C. Pehlevan MLT 17 78 0 19 May 2022
The large learning rate phase of deep learning: the catapult mechanism Aitor Lewkowycz Yasaman Bahri Ethan Dyer Jascha Narain Sohl-Dickstein Guy Gur-Ari ODL 150 232 0 04 Mar 2020
Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networks Blake Bordelon Abdulkadir Canatar C. Pehlevan 131 199 0 07 Feb 2020