ResearchTrend.AI
  • Communities
  • Connect sessions
  • AI calendar
  • Organizations
  • Join Slack
  • Contact Sales
Papers
Communities
Social Events
Terms and Conditions
Pricing
Contact Sales
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2026 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 1712.08969
  4. Cited By
Mean Field Residual Networks: On the Edge of Chaos

Mean Field Residual Networks: On the Edge of Chaos

Neural Information Processing Systems (NeurIPS), 2017
24 December 2017
Greg Yang
S. Schoenholz
ArXiv (abs)PDFHTML

Papers citing "Mean Field Residual Networks: On the Edge of Chaos"

30 / 130 papers shown
A Signal Propagation Perspective for Pruning Neural Networks at
  Initialization
A Signal Propagation Perspective for Pruning Neural Networks at InitializationInternational Conference on Learning Representations (ICLR), 2019
Namhoon Lee
Thalaiyasingam Ajanthan
Stephen Gould
Juil Sock
AAML
186
164
0
14 Jun 2019
The Normalization Method for Alleviating Pathological Sharpness in Wide
  Neural Networks
The Normalization Method for Alleviating Pathological Sharpness in Wide Neural NetworksNeural Information Processing Systems (NeurIPS), 2019
Ryo Karakida
S. Akaho
S. Amari
140
43
0
07 Jun 2019
A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth
  Trade-Off
A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-OffNeural Information Processing Systems (NeurIPS), 2019
Yaniv Blumenfeld
D. Gilboa
Daniel Soudry
MQ
193
14
0
03 Jun 2019
Exact Convergence Rates of the Neural Tangent Kernel in the Large Depth
  Limit
Exact Convergence Rates of the Neural Tangent Kernel in the Large Depth Limit
Soufiane Hayou
Arnaud Doucet
Judith Rousseau
403
5
0
31 May 2019
Infinitely deep neural networks as diffusion processes
Infinitely deep neural networks as diffusion processesInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2019
Stefano Peluchetti
Stefano Favaro
ODL
229
34
0
27 May 2019
Stabilize Deep ResNet with A Sharp Scaling Factor $τ$
Stabilize Deep ResNet with A Sharp Scaling Factor τττ
Huishuai Zhang
Da Yu
Mingyang Yi
Wei Chen
Tie-Yan Liu
395
11
0
17 Mar 2019
Mean-field Analysis of Batch Normalization
Mean-field Analysis of Batch Normalization
Ming-Bo Wei
J. Stokes
D. Schwab
MLT
92
8
0
06 Mar 2019
Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient
  Descent
Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent
Jaehoon Lee
Lechao Xiao
S. Schoenholz
Yasaman Bahri
Roman Novak
Jascha Narain Sohl-Dickstein
Jeffrey Pennington
639
1,215
0
18 Feb 2019
Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian
  Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation
Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation
Greg Yang
503
296
0
13 Feb 2019
Mean Field Limit of the Learning Dynamics of Multilayer Neural Networks
Mean Field Limit of the Learning Dynamics of Multilayer Neural Networks
Phan-Minh Nguyen
AI4CE
188
72
0
07 Feb 2019
Fixup Initialization: Residual Learning Without Normalization
Fixup Initialization: Residual Learning Without Normalization
Hongyi Zhang
Yann N. Dauphin
Tengyu Ma
ODLAI4CE
307
370
0
27 Jan 2019
Dynamical Isometry and a Mean Field Theory of LSTMs and GRUs
Dynamical Isometry and a Mean Field Theory of LSTMs and GRUs
D. Gilboa
B. Chang
Minmin Chen
Greg Yang
S. Schoenholz
Ed H. Chi
Jeffrey Pennington
225
43
0
25 Jan 2019
On the effect of the activation function on the distribution of hidden
  nodes in a deep network
On the effect of the activation function on the distribution of hidden nodes in a deep network
Philip M. Long
Hanie Sedghi
153
5
0
07 Jan 2019
NIPS - Not Even Wrong? A Systematic Review of Empirically Complete
  Demonstrations of Algorithmic Effectiveness in the Machine Learning and
  Artificial Intelligence Literature
NIPS - Not Even Wrong? A Systematic Review of Empirically Complete Demonstrations of Algorithmic Effectiveness in the Machine Learning and Artificial Intelligence Literature
Franz J. Király
Bilal A. Mateen
R. Sonabend
192
10
0
18 Dec 2018
Characterizing Well-Behaved vs. Pathological Deep Neural Networks
Characterizing Well-Behaved vs. Pathological Deep Neural Networks
Mitchell Stern
184
0
0
07 Nov 2018
Critical initialisation for deep signal propagation in noisy rectifier
  neural networks
Critical initialisation for deep signal propagation in noisy rectifier neural networks
Arnu Pretorius
Elan Van Biljon
Steve Kroon
Herman Kamper
153
17
0
01 Nov 2018
On the Convergence Rate of Training Recurrent Neural Networks
On the Convergence Rate of Training Recurrent Neural Networks
Zeyuan Allen-Zhu
Yuanzhi Li
Zhao Song
574
200
0
29 Oct 2018
Bayesian Deep Convolutional Networks with Many Channels are Gaussian
  Processes
Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes
Roman Novak
Lechao Xiao
Jaehoon Lee
Yasaman Bahri
Greg Yang
Jiri Hron
Daniel A. Abolafia
Jeffrey Pennington
Jascha Narain Sohl-Dickstein
UQCVBDL
326
324
0
11 Oct 2018
Dynamical Isometry is Achieved in Residual Networks in a Universal Way
  for any Activation Function
Dynamical Isometry is Achieved in Residual Networks in a Universal Way for any Activation FunctionInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2018
W. Tarnowski
P. Warchol
Stanislaw Jastrzebski
Jacek Tabor
M. Nowak
192
40
0
24 Sep 2018
Fisher Information and Natural Gradient Learning of Random Deep Networks
Fisher Information and Natural Gradient Learning of Random Deep Networks
S. Amari
Ryo Karakida
Masafumi Oizumi
179
50
0
22 Aug 2018
Statistical Neurodynamics of Deep Networks: Geometry of Signal Spaces
Statistical Neurodynamics of Deep Networks: Geometry of Signal Spaces
S. Amari
Ryo Karakida
Masafumi Oizumi
96
9
0
22 Aug 2018
Spectrum concentration in deep residual learning: a free probability
  approach
Spectrum concentration in deep residual learning: a free probability approach
Zenan Ling
Xing He
Robert C. Qiu
192
22
0
31 Jul 2018
Initialization of ReLUs for Dynamical Isometry
Initialization of ReLUs for Dynamical Isometry
R. Burkholz
Alina Dubatovka
ODLAI4CE
93
3
0
17 Jun 2018
Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables
  Signal Propagation in Recurrent Neural Networks
Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networks
Minmin Chen
Jeffrey Pennington
S. Schoenholz
SyDaAI4CE
182
124
0
14 Jun 2018
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train
  10,000-Layer Vanilla Convolutional Neural Networks
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks
Lechao Xiao
Yasaman Bahri
Jascha Narain Sohl-Dickstein
S. Schoenholz
Jeffrey Pennington
535
368
0
14 Jun 2018
Universal Statistics of Fisher Information in Deep Neural Networks: Mean
  Field Approach
Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach
Ryo Karakida
S. Akaho
S. Amari
FedML
524
162
0
04 Jun 2018
The Nonlinearity Coefficient - Predicting Generalization in Deep Neural
  Networks
The Nonlinearity Coefficient - Predicting Generalization in Deep Neural Networks
George Philipp
J. Carbonell
101
14
0
01 Jun 2018
On the Selection of Initialization and Activation Function for Deep
  Neural Networks
On the Selection of Initialization and Activation Function for Deep Neural Networks
Soufiane Hayou
Arnaud Doucet
Judith Rousseau
ODL
121
77
0
21 May 2018
How to Start Training: The Effect of Initialization and Architecture
How to Start Training: The Effect of Initialization and Architecture
Boris Hanin
David Rolnick
256
273
0
05 Mar 2018
The exploding gradient problem demystified - definition, prevalence,
  impact, origin, tradeoffs, and solutions
The exploding gradient problem demystified - definition, prevalence, impact, origin, tradeoffs, and solutions
George Philipp
Basel Alomair
J. Carbonell
ODL
321
49
0
15 Dec 2017
Previous
123