Beyond Random Matrix Theory for Deep Networks

13 June 2020

Papers citing "Beyond Random Matrix Theory for Deep Networks"

18 / 18 papers shown

Title
Average-case Acceleration Through Spectral Density Estimation Fabian Pedregosa Damien Scieur 21 12 0 12 Feb 2020
Deep Curvature Suite Diego Granziol Xingchen Wan T. Garipov 3DV 26 12 0 20 Dec 2019
Limitations of the Empirical Fisher Approximation for Natural Gradient Descent Frederik Kunstner Lukas Balles Philipp Hennig 58 212 0 29 May 2019
An Investigation into Neural Net Optimization via Hessian Eigenvalue Density Behrooz Ghorbani Shankar Krishnan Ying Xiao ODL 42 320 0 29 Jan 2019
Measurements of Three-Level Hierarchical Structure in the Outliers in the Spectrum of Deepnet Hessians Vardan Papyan 47 87 0 24 Jan 2019
Averaging Weights Leads to Wider Optima and Better Generalization Pavel Izmailov Dmitrii Podoprikhin T. Garipov Dmitry Vetrov A. Wilson FedML MoMe 93 1,643 0 14 Mar 2018
Essentially No Barriers in Neural Network Energy Landscape Felix Dräxler K. Veschgini M. Salmhofer Fred Hamprecht MoMe 97 430 0 02 Mar 2018
Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs T. Garipov Pavel Izmailov Dmitrii Podoprikhin Dmitry Vetrov A. Wilson UQCV 61 746 0 27 Feb 2018
Empirical Analysis of the Hessian of Over-Parametrized Neural Networks Levent Sagun Utku Evci V. U. Güney Yann N. Dauphin Léon Bottou 41 416 0 14 Jun 2017
The loss surface of deep and wide neural networks Quynh N. Nguyen Matthias Hein ODL 89 284 0 26 Apr 2017
Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond Levent Sagun Léon Bottou Yann LeCun UQCV 71 233 0 22 Nov 2016
Cleaning large correlation matrices: tools from random matrix theory J. Bun J. Bouchaud M. Potters 60 263 0 25 Oct 2016
Deep Learning without Poor Local Minima Kenji Kawaguchi ODL 165 922 0 23 May 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.4K 192,638 0 10 Dec 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 328 43,154 0 11 Feb 2015
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 230 1,191 0 30 Nov 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 954 99,991 0 04 Sep 2014
Distributions of Angles in Random Packing on Spheres Tony Cai Jianqing Fan Tiefeng Jiang 102 183 0 02 Jun 2013