The Spectrum of Fisher Information of Deep Networks Achieving Dynamical
Isometry

v1v2v3v4 (latest)

The Spectrum of Fisher Information of Deep Networks Achieving Dynamical Isometry

14 June 2020

Tomohiro Hayase

ArXiv (abs)PDF HTML

Papers citing "The Spectrum of Fisher Information of Deep Networks Achieving Dynamical Isometry"

7 / 7 papers shown

Title
Efficient Activation Function Optimization through Surrogate Modeling G. Bingham Risto Miikkulainen 142 2 0 13 Jan 2023
Dynamical Isometry for Residual Networks Advait Gadhikar R. Burkholz ODL AI4CE 124 2 0 05 Oct 2022
Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks Zhichao Wang Yizhe Zhu 141 22 0 20 Sep 2021
Asymptotic Freeness of Layerwise Jacobians Caused by Invariance of Multilayer Perceptron: The Haar Orthogonal Case B. Collins Tomohiro Hayase 101 8 0 24 Mar 2021
Exact Convergence Rates of the Neural Tangent Kernel in the Large Depth Limit Soufiane Hayou Arnaud Doucet Judith Rousseau 151 4 0 31 May 2019
A Geometric Modeling of Occam's Razor in Deep Learning Ke Sun Frank Nielsen 109 5 0 27 May 2019
Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach Ryo Karakida S. Akaho S. Amari FedML 344 152 0 04 Jun 2018