ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 1806.01316
  4. Cited By
Universal Statistics of Fisher Information in Deep Neural Networks: Mean
  Field Approach

Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach

4 June 2018
Ryo Karakida
S. Akaho
S. Amari
    FedML
ArXivPDFHTML

Papers citing "Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach"

46 / 96 papers shown
Title
Towards NNGP-guided Neural Architecture Search
Towards NNGP-guided Neural Architecture Search
Daniel S. Park
Jaehoon Lee
Daiyi Peng
Yuan Cao
Jascha Narain Sohl-Dickstein
BDL
18
32
0
11 Nov 2020
The power of quantum neural networks
The power of quantum neural networks
Amira Abbas
David Sutter
Christa Zoufal
Aurelien Lucchi
Alessio Figalli
Stefan Woerner
13
725
0
30 Oct 2020
Dissecting Hessian: Understanding Common Structure of Hessian in Neural
  Networks
Dissecting Hessian: Understanding Common Structure of Hessian in Neural Networks
Yikai Wu
Xingyu Zhu
Chenwei Wu
Annie Wang
Rong Ge
13
42
0
08 Oct 2020
Understanding Approximate Fisher Information for Fast Convergence of
  Natural Gradient Descent in Wide Neural Networks
Understanding Approximate Fisher Information for Fast Convergence of Natural Gradient Descent in Wide Neural Networks
Ryo Karakida
Kazuki Osawa
14
25
0
02 Oct 2020
Implicit Gradient Regularization
Implicit Gradient Regularization
David Barrett
Benoit Dherin
14
146
0
23 Sep 2020
Traces of Class/Cross-Class Structure Pervade Deep Learning Spectra
Traces of Class/Cross-Class Structure Pervade Deep Learning Spectra
V. Papyan
14
76
0
27 Aug 2020
Implicit Regularization via Neural Feature Alignment
Implicit Regularization via Neural Feature Alignment
A. Baratin
Thomas George
César Laurent
R. Devon Hjelm
Guillaume Lajoie
Pascal Vincent
Simon Lacoste-Julien
18
6
0
03 Aug 2020
Finite Versus Infinite Neural Networks: an Empirical Study
Finite Versus Infinite Neural Networks: an Empirical Study
Jaehoon Lee
S. Schoenholz
Jeffrey Pennington
Ben Adlam
Lechao Xiao
Roman Novak
Jascha Narain Sohl-Dickstein
17
207
0
31 Jul 2020
Bayesian Deep Ensembles via the Neural Tangent Kernel
Bayesian Deep Ensembles via the Neural Tangent Kernel
Bobby He
Balaji Lakshminarayanan
Yee Whye Teh
BDL
UQCV
9
116
0
11 Jul 2020
When Does Preconditioning Help or Hurt Generalization?
When Does Preconditioning Help or Hurt Generalization?
S. Amari
Jimmy Ba
Roger C. Grosse
Xuechen Li
Atsushi Nitanda
Taiji Suzuki
Denny Wu
Ji Xu
34
32
0
18 Jun 2020
The Spectrum of Fisher Information of Deep Networks Achieving Dynamical
  Isometry
The Spectrum of Fisher Information of Deep Networks Achieving Dynamical Isometry
Tomohiro Hayase
Ryo Karakida
27
7
0
14 Jun 2020
Non-convergence of stochastic gradient descent in the training of deep
  neural networks
Non-convergence of stochastic gradient descent in the training of deep neural networks
Patrick Cheridito
Arnulf Jentzen
Florian Rossmannek
14
37
0
12 Jun 2020
Spectra of the Conjugate Kernel and Neural Tangent Kernel for
  linear-width neural networks
Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networks
Z. Fan
Zhichao Wang
44
71
0
25 May 2020
Using a thousand optimization tasks to learn hyperparameter search
  strategies
Using a thousand optimization tasks to learn hyperparameter search strategies
Luke Metz
Niru Maheswaranathan
Ruoxi Sun
C. Freeman
Ben Poole
Jascha Narain Sohl-Dickstein
20
45
0
27 Feb 2020
Layer-wise Conditioning Analysis in Exploring the Learning Dynamics of
  DNNs
Layer-wise Conditioning Analysis in Exploring the Learning Dynamics of DNNs
Lei Huang
Jie Qin
Li Liu
Fan Zhu
Ling Shao
AI4CE
28
11
0
25 Feb 2020
Towards Query-Efficient Black-Box Adversary with Zeroth-Order Natural
  Gradient Descent
Towards Query-Efficient Black-Box Adversary with Zeroth-Order Natural Gradient Descent
Pu Zhao
Pin-Yu Chen
Siyue Wang
X. Lin
AAML
8
36
0
18 Feb 2020
On the infinite width limit of neural networks with a standard
  parameterization
On the infinite width limit of neural networks with a standard parameterization
Jascha Narain Sohl-Dickstein
Roman Novak
S. Schoenholz
Jaehoon Lee
24
47
0
21 Jan 2020
Any Target Function Exists in a Neighborhood of Any Sufficiently Wide
  Random Network: A Geometrical Perspective
Any Target Function Exists in a Neighborhood of Any Sufficiently Wide Random Network: A Geometrical Perspective
S. Amari
19
12
0
20 Jan 2020
Disentangling Trainability and Generalization in Deep Neural Networks
Disentangling Trainability and Generalization in Deep Neural Networks
Lechao Xiao
Jeffrey Pennington
S. Schoenholz
6
34
0
30 Dec 2019
Neural Tangents: Fast and Easy Infinite Neural Networks in Python
Neural Tangents: Fast and Easy Infinite Neural Networks in Python
Roman Novak
Lechao Xiao
Jiri Hron
Jaehoon Lee
Alexander A. Alemi
Jascha Narain Sohl-Dickstein
S. Schoenholz
27
224
0
05 Dec 2019
Information-Theoretic Local Minima Characterization and Regularization
Information-Theoretic Local Minima Characterization and Regularization
Zhiwei Jia
Hao Su
21
19
0
19 Nov 2019
Sparsification as a Remedy for Staleness in Distributed Asynchronous SGD
Sparsification as a Remedy for Staleness in Distributed Asynchronous SGD
Rosa Candela
Giulio Franzese
Maurizio Filippone
Pietro Michiardi
15
1
0
21 Oct 2019
Neural Spectrum Alignment: Empirical Study
Neural Spectrum Alignment: Empirical Study
Dmitry Kopitkov
Vadim Indelman
27
14
0
19 Oct 2019
Pathological spectra of the Fisher information metric and its variants
  in deep neural networks
Pathological spectra of the Fisher information metric and its variants in deep neural networks
Ryo Karakida
S. Akaho
S. Amari
17
27
0
14 Oct 2019
The asymptotic spectrum of the Hessian of DNN throughout training
The asymptotic spectrum of the Hessian of DNN throughout training
Arthur Jacot
Franck Gabriel
Clément Hongler
11
34
0
01 Oct 2019
Deep Learning Theory Review: An Optimal Control and Dynamical Systems
  Perspective
Deep Learning Theory Review: An Optimal Control and Dynamical Systems Perspective
Guan-Horng Liu
Evangelos A. Theodorou
AI4CE
11
71
0
28 Aug 2019
A Fine-Grained Spectral Perspective on Neural Networks
A Fine-Grained Spectral Perspective on Neural Networks
Greg Yang
Hadi Salman
22
110
0
24 Jul 2019
Deep network as memory space: complexity, generalization, disentangled
  representation and interpretability
Deep network as memory space: complexity, generalization, disentangled representation and interpretability
X. Dong
L. Zhou
23
1
0
12 Jul 2019
Order and Chaos: NTK views on DNN Normalization, Checkerboard and
  Boundary Artifacts
Order and Chaos: NTK views on DNN Normalization, Checkerboard and Boundary Artifacts
Arthur Jacot
Franck Gabriel
François Ged
Clément Hongler
11
23
0
11 Jul 2019
The Normalization Method for Alleviating Pathological Sharpness in Wide
  Neural Networks
The Normalization Method for Alleviating Pathological Sharpness in Wide Neural Networks
Ryo Karakida
S. Akaho
S. Amari
21
39
0
07 Jun 2019
Exact Convergence Rates of the Neural Tangent Kernel in the Large Depth
  Limit
Exact Convergence Rates of the Neural Tangent Kernel in the Large Depth Limit
Soufiane Hayou
Arnaud Doucet
Judith Rousseau
16
4
0
31 May 2019
Limitations of the Empirical Fisher Approximation for Natural Gradient
  Descent
Limitations of the Empirical Fisher Approximation for Natural Gradient Descent
Frederik Kunstner
Lukas Balles
Philipp Hennig
21
207
0
29 May 2019
A Geometric Modeling of Occam's Razor in Deep Learning
A Geometric Modeling of Occam's Razor in Deep Learning
Ke Sun
Frank Nielsen
11
4
0
27 May 2019
The Effect of Network Width on Stochastic Gradient Descent and
  Generalization: an Empirical Study
The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Study
Daniel S. Park
Jascha Narain Sohl-Dickstein
Quoc V. Le
Samuel L. Smith
14
57
0
09 May 2019
Mean-field Analysis of Batch Normalization
Mean-field Analysis of Batch Normalization
Ming-Bo Wei
J. Stokes
D. Schwab
MLT
25
8
0
06 Mar 2019
Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian
  Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation
Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation
Greg Yang
11
282
0
13 Feb 2019
Large-Scale Distributed Second-Order Optimization Using
  Kronecker-Factored Approximate Curvature for Deep Convolutional Neural
  Networks
Large-Scale Distributed Second-Order Optimization Using Kronecker-Factored Approximate Curvature for Deep Convolutional Neural Networks
Kazuki Osawa
Yohei Tsuji
Yuichiro Ueno
Akira Naruse
Rio Yokota
Satoshi Matsuoka
ODL
28
95
0
29 Nov 2018
Measuring the Effects of Data Parallelism on Neural Network Training
Measuring the Effects of Data Parallelism on Neural Network Training
Christopher J. Shallue
Jaehoon Lee
J. Antognini
J. Mamou
J. Ketterling
Yao Wang
35
407
0
08 Nov 2018
Information Geometry of Orthogonal Initializations and Training
Information Geometry of Orthogonal Initializations and Training
Piotr A. Sokól
Il-Su Park
AI4CE
72
16
0
09 Oct 2018
Fisher Information and Natural Gradient Learning of Random Deep Networks
Fisher Information and Natural Gradient Learning of Random Deep Networks
S. Amari
Ryo Karakida
Masafumi Oizumi
14
34
0
22 Aug 2018
Neural Tangent Kernel: Convergence and Generalization in Neural Networks
Neural Tangent Kernel: Convergence and Generalization in Neural Networks
Arthur Jacot
Franck Gabriel
Clément Hongler
14
3,098
0
20 Jun 2018
Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables
  Signal Propagation in Recurrent Neural Networks
Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networks
Minmin Chen
Jeffrey Pennington
S. Schoenholz
SyDa
AI4CE
6
114
0
14 Jun 2018
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train
  10,000-Layer Vanilla Convolutional Neural Networks
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks
Lechao Xiao
Yasaman Bahri
Jascha Narain Sohl-Dickstein
S. Schoenholz
Jeffrey Pennington
224
348
0
14 Jun 2018
Fisher-Rao Metric, Geometry, and Complexity of Neural Networks
Fisher-Rao Metric, Geometry, and Complexity of Neural Networks
Tengyuan Liang
T. Poggio
Alexander Rakhlin
J. Stokes
25
224
0
05 Nov 2017
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp
  Minima
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
N. Keskar
Dheevatsa Mudigere
J. Nocedal
M. Smelyanskiy
P. T. P. Tang
ODL
281
2,889
0
15 Sep 2016
Norm-Based Capacity Control in Neural Networks
Norm-Based Capacity Control in Neural Networks
Behnam Neyshabur
Ryota Tomioka
Nathan Srebro
119
577
0
27 Feb 2015
Previous
12