Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
1712.08969
Cited By
Mean Field Residual Networks: On the Edge of Chaos
Neural Information Processing Systems (NeurIPS), 2017
24 December 2017
Greg Yang
S. Schoenholz
Re-assign community
ArXiv (abs)
PDF
HTML
Papers citing
"Mean Field Residual Networks: On the Edge of Chaos"
50 / 130 papers shown
When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations
International Conference on Learning Representations (ICLR), 2021
Xiangning Chen
Cho-Jui Hsieh
Boqing Gong
ViT
371
375
0
03 Jun 2021
Tensor Programs IIb: Architectural Universality of Neural Tangent Kernel Training Dynamics
International Conference on Machine Learning (ICML), 2021
Greg Yang
Etai Littwin
161
76
0
08 May 2021
Initialization and Regularization of Factorized Neural Layers
International Conference on Learning Representations (ICLR), 2021
M. Khodak
Neil A. Tenenholtz
Lester W. Mackey
Nicolò Fusi
420
68
0
03 May 2021
ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training
International Conference on Machine Learning (ICML), 2021
Jianfei Chen
Lianmin Zheng
Z. Yao
Yi Xu
Ion Stoica
Michael W. Mahoney
Joseph E. Gonzalez
MQ
214
82
0
29 Apr 2021
Towards Deepening Graph Neural Networks: A GNTK-based Optimization Perspective
International Conference on Learning Representations (ICLR), 2021
Wei Huang
Yayong Li
Weitao Du
Jie Yin
R. Xu
Ling-Hao Chen
Miao Zhang
208
19
0
03 Mar 2021
Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective
International Conference on Learning Representations (ICLR), 2021
Wuyang Chen
Xinyu Gong
Zinan Lin
OOD
495
275
0
23 Feb 2021
Formalising the Use of the Activation Function in Neural Inference
Complex Systems (CS), 2021
D. A. R. Sakthivadivel
189
4
0
02 Feb 2021
Characterizing signal propagation to close the performance gap in unnormalized ResNets
International Conference on Learning Representations (ICLR), 2021
Andrew Brock
Soham De
Samuel L. Smith
396
134
0
21 Jan 2021
Advances in Electron Microscopy with Deep Learning
Jeffrey M. Ede
703
3
0
04 Jan 2021
Analyzing Finite Neural Networks: Can We Trust Neural Tangent Kernel Theory?
Mariia Seleznova
Gitta Kutyniok
AAML
245
36
0
08 Dec 2020
Feature Learning in Infinite-Width Neural Networks
Greg Yang
J. E. Hu
MLT
424
182
0
30 Nov 2020
Towards NNGP-guided Neural Architecture Search
Daniel S. Park
Jaehoon Lee
Daiyi Peng
Yuan Cao
Jascha Narain Sohl-Dickstein
BDL
184
34
0
11 Nov 2020
Stable ResNet
International Conference on Artificial Intelligence and Statistics (AISTATS), 2020
Soufiane Hayou
Eugenio Clerico
Bo He
George Deligiannidis
Arnaud Doucet
Judith Rousseau
ODL
SSeg
158
60
0
24 Oct 2020
BYOL works even without batch statistics
Pierre Harvey Richemond
Jean-Bastien Grill
Florent Altché
Corentin Tallec
Florian Strub
...
Samuel L. Smith
Soham De
Razvan Pascanu
Bilal Piot
Michal Valko
SSL
481
120
0
20 Oct 2020
Exploring the Uncertainty Properties of Neural Networks' Implicit Priors in the Infinite-Width Limit
Ben Adlam
Jaehoon Lee
Lechao Xiao
Jeffrey Pennington
Jasper Snoek
UQCV
BDL
208
16
0
14 Oct 2020
Understanding Approximate Fisher Information for Fast Convergence of Natural Gradient Descent in Wide Neural Networks
Neural Information Processing Systems (NeurIPS), 2020
Ryo Karakida
Kazuki Osawa
269
31
0
02 Oct 2020
Tensor Programs III: Neural Matrix Laws
Greg Yang
355
53
0
22 Sep 2020
Review: Deep Learning in Electron Microscopy
Jeffrey M. Ede
949
90
0
17 Sep 2020
Continuous-in-Depth Neural Networks
A. Queiruga
N. Benjamin Erichson
D. Taylor
Michael W. Mahoney
289
54
0
05 Aug 2020
Finite Versus Infinite Neural Networks: an Empirical Study
Neural Information Processing Systems (NeurIPS), 2020
Jaehoon Lee
S. Schoenholz
Jeffrey Pennington
Ben Adlam
Lechao Xiao
Roman Novak
Jascha Narain Sohl-Dickstein
310
228
0
31 Jul 2020
Doubly infinite residual neural networks: a diffusion process approach
Stefano Peluchetti
Stefano Favaro
124
2
0
07 Jul 2020
Beyond Signal Propagation: Is Feature Diversity Necessary in Deep Neural Network Initialization?
Yaniv Blumenfeld
D. Gilboa
Daniel Soudry
ODL
199
16
0
02 Jul 2020
Tensor Programs II: Neural Tangent Kernel for Any Architecture
Greg Yang
473
156
0
25 Jun 2020
Fractional moment-preserving initialization schemes for training deep neural networks
Mert Gurbuzbalaban
Yuanhan Hu
232
3
0
25 May 2020
Understanding the Difficulty of Training Transformers
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020
Liyuan Liu
Xiaodong Liu
Jianfeng Gao
Weizhu Chen
Jiawei Han
AI4CE
285
286
0
17 Apr 2020
On the Neural Tangent Kernel of Deep Networks with Orthogonal Initialization
International Joint Conference on Artificial Intelligence (IJCAI), 2020
Wei Huang
Weitao Du
R. Xu
175
40
0
13 Apr 2020
On Infinite-Width Hypernetworks
Etai Littwin
Tomer Galanti
Lior Wolf
Greg Yang
457
11
0
27 Mar 2020
Towards a General Theory of Infinite-Width Limits of Neural Classifiers
International Conference on Machine Learning (ICML), 2020
Eugene Golikov
AI4CE
126
9
0
12 Mar 2020
ReZero is All You Need: Fast Convergence at Large Depth
Conference on Uncertainty in Artificial Intelligence (UAI), 2020
Thomas C. Bachlechner
Bodhisattwa Prasad Majumder
H. H. Mao
G. Cottrell
Julian McAuley
AI4CE
386
329
0
10 Mar 2020
Correlated Initialization for Correlated Data
Neural Processing Letters (NPL), 2020
Johannes Schneider
186
6
0
09 Mar 2020
Convolutional Spectral Kernel Learning
Artificial Intelligence (AIJ), 2020
Jian Li
Yong Liu
Weiping Wang
BDL
92
5
0
28 Feb 2020
Using a thousand optimization tasks to learn hyperparameter search strategies
Luke Metz
Niru Maheswaranathan
Ruoxi Sun
C. Freeman
Ben Poole
Jascha Narain Sohl-Dickstein
328
50
0
27 Feb 2020
Robust Pruning at Initialization
International Conference on Learning Representations (ICLR), 2020
Soufiane Hayou
Jean-François Ton
Arnaud Doucet
Yee Whye Teh
186
49
0
19 Feb 2020
On the distance between two neural networks and the stability of learning
Neural Information Processing Systems (NeurIPS), 2020
Jeremy Bernstein
Arash Vahdat
Yisong Yue
Xuan Li
ODL
498
69
0
09 Feb 2020
On Random Kernels of Residual Architectures
Etai Littwin
Tomer Galanti
Lior Wolf
244
4
0
28 Jan 2020
Disentangling Trainability and Generalization in Deep Neural Networks
Lechao Xiao
Jeffrey Pennington
S. Schoenholz
199
34
0
30 Dec 2019
Towards Efficient Training for Neural Network Quantization
Qing Jin
Linjie Yang
Zhenyu A. Liao
MQ
240
42
0
21 Dec 2019
Mean field theory for deep dropout networks: digging up gradient backpropagation deeply
European Conference on Artificial Intelligence (ECAI), 2019
Wei Huang
R. Xu
Weitao Du
Yutian Zeng
Yunce Zhao
157
6
0
19 Dec 2019
Optimization for deep learning: theory and algorithms
Tian Ding
ODL
340
178
0
19 Dec 2019
Is Feature Diversity Necessary in Neural Network Initialization?
Yaniv Blumenfeld
D. Gilboa
Daniel Soudry
147
0
0
11 Dec 2019
Neural Tangents: Fast and Easy Infinite Neural Networks in Python
International Conference on Learning Representations (ICLR), 2019
Roman Novak
Lechao Xiao
Jiri Hron
Jaehoon Lee
Alexander A. Alemi
Jascha Narain Sohl-Dickstein
S. Schoenholz
221
249
0
05 Dec 2019
Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes
Neural Information Processing Systems (NeurIPS), 2019
Greg Yang
483
221
0
28 Oct 2019
Pathological spectra of the Fisher information metric and its variants in deep neural networks
Neural Computation (Neural Comput.), 2019
Ryo Karakida
S. Akaho
S. Amari
203
32
0
14 Oct 2019
Large Deviation Analysis of Function Sensitivity in Random Deep Neural Networks
Bo Li
D. Saad
136
12
0
13 Oct 2019
On the expected behaviour of noise regularised deep neural networks as Gaussian processes
Pattern Recognition Letters (PR), 2019
Arnu Pretorius
Herman Kamper
Steve Kroon
175
9
0
12 Oct 2019
The Expressivity and Training of Deep Neural Networks: toward the Edge of Chaos?
Gege Zhang
Gang-cheng Li
Ningwei Shen
Weidong Zhang
167
7
0
11 Oct 2019
Deep Learning Theory Review: An Optimal Control and Dynamical Systems Perspective
Guan-Horng Liu
Evangelos A. Theodorou
AI4CE
303
74
0
28 Aug 2019
Almost Sure Asymptotic Freeness of Neural Network Jacobian with Orthogonal Weights
Tomohiro Hayase
112
0
0
11 Aug 2019
A Fine-Grained Spectral Perspective on Neural Networks
Greg Yang
Hadi Salman
379
117
0
24 Jul 2019
Order and Chaos: NTK views on DNN Normalization, Checkerboard and Boundary Artifacts
Arthur Jacot
Franck Gabriel
François Ged
Clément Hongler
141
24
0
11 Jul 2019
Previous
1
2
3
Next
Page 2 of 3
