Products of Many Large Random Matrices and Gradients in Deep Neural Networks

14 December 2018

Papers citing "Products of Many Large Random Matrices and Gradients in Deep Neural Networks"

24 / 24 papers shown

Title
Don't be lazy: CompleteP enables compute-efficient deep transformers Nolan Dey Bin Claire Zhang Lorenzo Noci Mufan Li Blake Bordelon Shane Bergsma Cengiz Pehlevan Boris Hanin Joel Hestness 44 1 0 02 May 2025
Deep Neural Nets as Hamiltonians Mike Winer Boris Hanin 232 0 0 31 Mar 2025
Feature Learning Beyond the Edge of Stability Dávid Terjék MLT 51 0 0 18 Feb 2025
Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networks Blake Bordelon Cengiz Pehlevan MLT 43 29 0 06 Apr 2023
Injectivity of ReLU networks: perspectives from statistical physics Antoine Maillard Afonso S. Bandeira David Belius Ivan Dokmanić S. Nakajima 33 5 0 27 Feb 2023
Width and Depth Limits Commute in Residual Networks Soufiane Hayou Greg Yang 50 14 0 01 Feb 2023
Expected Gradients of Maxout Networks and Consequences to Parameter Initialization Hanna Tseran Guido Montúfar ODL 35 0 0 17 Jan 2023
Effects of Data Geometry in Early Deep Learning Saket Tiwari George Konidaris 82 7 0 29 Dec 2022
Infinite-width limit of deep linear neural networks Lénaïc Chizat Maria Colombo Xavier Fernández-Real Alessio Figalli 31 14 0 29 Nov 2022
Spectral Evolution and Invariance in Linear-width Neural Networks Zhichao Wang A. Engel Anand D. Sarwate Ioana Dumitriu Tony Chiang 45 14 0 11 Nov 2022
Deep Linear Networks for Matrix Completion -- An Infinite Depth Limit Nadav Cohen Govind Menon Zsolt Veraszto ODL 29 7 0 22 Oct 2022
Meta-Principled Family of Hyperparameter Scaling Strategies Sho Yaida 58 16 0 10 Oct 2022
On skip connections and normalisation layers in deep optimisation L. MacDonald Jack Valmadre Hemanth Saratchandran Simon Lucey ODL 37 1 0 10 Oct 2022
The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initialization Mufan Li Mihai Nica Daniel M. Roy 55 37 0 06 Jun 2022
Random Neural Networks in the Infinite Width Limit as Gaussian Processes Boris Hanin BDL 37 44 0 04 Jul 2021
Precise characterization of the prior predictive distribution of deep ReLU networks Lorenzo Noci Gregor Bachmann Kevin Roth Sebastian Nowozin Thomas Hofmann BDL UQCV 34 32 0 11 Jun 2021
The Future is Log-Gaussian: ResNets and Their Infinite-Depth-and-Width Limit at Initialization Mufan Li Mihai Nica Daniel M. Roy 43 33 0 07 Jun 2021
Asymptotic Freeness of Layerwise Jacobians Caused by Invariance of Multilayer Perceptron: The Haar Orthogonal Case B. Collins Tomohiro Hayase 22 7 0 24 Mar 2021
Deep ReLU Networks Preserve Expected Length Boris Hanin Ryan Jeong David Rolnick 29 14 0 21 Feb 2021
Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networks Quynh N. Nguyen Marco Mondelli Guido Montúfar 25 81 0 21 Dec 2020
The Spectrum of Fisher Information of Deep Networks Achieving Dynamical Isometry Tomohiro Hayase Ryo Karakida 29 7 0 14 Jun 2020
Asymptotics of Wide Networks from Feynman Diagrams Ethan Dyer Guy Gur-Ari 32 114 0 25 Sep 2019
Finite Depth and Width Corrections to the Neural Tangent Kernel Boris Hanin Mihai Nica MDE 30 149 0 13 Sep 2019
Eigenvalue distribution of nonlinear models of random matrices L. Benigni Sandrine Péché 35 27 0 05 Apr 2019