High-dimensional dynamics of generalization error in neural networks

10 October 2017

Papers citing "High-dimensional dynamics of generalization error in neural networks"

46 / 296 papers shown

Title
Student Specialization in Deep ReLU Networks With Finite Width and Input Dimension Yuandong Tian MLT 6 8 0 30 Sep 2019
Modelling the influence of data structure on learning in neural networks: the hidden manifold model Sebastian Goldt M. Mézard Florent Krzakala Lenka Zdeborová BDL 21 51 0 25 Sep 2019
The generalization error of random features regression: Precise asymptotics and double descent curve Song Mei Andrea Montanari 39 624 0 14 Aug 2019
Disentangling feature and lazy training in deep neural networks Mario Geiger S. Spigler Arthur Jacot M. Wyart 13 17 0 19 Jun 2019
Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setup Sebastian Goldt Madhu S. Advani Andrew M. Saxe Florent Krzakala Lenka Zdeborová MLT 14 140 0 18 Jun 2019
Understanding overfitting peaks in generalization error: Analytical risk curves for $l_2$ and $l_1$ penalized interpolation P. Mitra 10 50 0 09 Jun 2019
Dimensionality compression and expansion in Deep Neural Networks Stefano Recanatesi M. Farrell Madhu S. Advani Timothy Moore Guillaume Lajoie E. Shea-Brown 13 72 0 02 Jun 2019
Implicit Regularization in Deep Matrix Factorization Sanjeev Arora Nadav Cohen Wei Hu Yuping Luo AI4CE 24 491 0 31 May 2019
Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks Guodong Zhang James Martens Roger C. Grosse ODL 8 124 0 27 May 2019
Meta-learners' learning dynamics are unlike learners' Neil C. Rabinowitz OffRL 17 16 0 03 May 2019
Similarity of Neural Network Representations Revisited Simon Kornblith Mohammad Norouzi Honglak Lee Geoffrey E. Hinton 13 1,355 0 01 May 2019
Implicit Regularization of Discrete Gradient Dynamics in Linear Neural Networks Gauthier Gidel Francis R. Bach Simon Lacoste-Julien AI4CE 6 150 0 30 Apr 2019
Layer Dynamics of Linearised Neural Nets Saurav Basu Koyel Mukherjee Shrihari Vasudevan AI4CE 6 1 0 24 Apr 2019
Surprises in High-Dimensional Ridgeless Least Squares Interpolation Trevor Hastie Andrea Montanari Saharon Rosset R. Tibshirani 18 726 0 19 Mar 2019
Two models of double descent for weak features M. Belkin Daniel J. Hsu Ji Xu 12 375 0 18 Mar 2019
SSN: Learning Sparse Switchable Normalization via SparsestMax Wenqi Shao Jiamin Ren Jingyu Li Ruimao Zhang Yudian Li Xiaogang Wang Ping Luo 18 56 0 09 Mar 2019
Critical initialisation in continuous approximations of binary neural networks G. Stamatescu Federica Gerace C. Lucibello I. Fuss L. White 17 0 0 01 Feb 2019
Numerically Recovering the Critical Points of a Deep Linear Autoencoder Charles G. Frye Neha S. Wadia M. DeWeese K. Bouchard 14 6 0 29 Jan 2019
Generalisation dynamics of online learning in over-parameterised neural networks Sebastian Goldt Madhu S. Advani Andrew M. Saxe Florent Krzakala Lenka Zdeborová 17 14 0 25 Jan 2019
A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks Umut Simsekli Levent Sagun Mert Gurbuzbalaban 17 237 0 18 Jan 2019
Scaling description of generalization with number of parameters in deep learning Mario Geiger Arthur Jacot S. Spigler Franck Gabriel Levent Sagun Stéphane dÁscoli Giulio Biroli Clément Hongler M. Wyart 36 194 0 06 Jan 2019
Reconciling modern machine learning practice and the bias-variance trade-off M. Belkin Daniel J. Hsu Siyuan Ma Soumik Mandal 20 1,608 0 28 Dec 2018
An Empirical Study of Example Forgetting during Deep Neural Network Learning Mariya Toneva Alessandro Sordoni Rémi Tachet des Combes Adam Trischler Yoshua Bengio Geoffrey J. Gordon 29 711 0 12 Dec 2018
Gradient Descent Happens in a Tiny Subspace Guy Gur-Ari Daniel A. Roberts Ethan Dyer 14 229 0 12 Dec 2018
Shared Representational Geometry Across Neural Networks Qihong Lu Po-Hsuan Chen Jonathan W. Pillow Peter J. Ramadge K. A. Norman Uri Hasson OOD 10 11 0 28 Nov 2018
A jamming transition from under- to over-parametrization affects loss landscape and generalization S. Spigler Mario Geiger Stéphane dÁscoli Levent Sagun Giulio Biroli M. Wyart 17 151 0 22 Oct 2018
A Modern Take on the Bias-Variance Tradeoff in Neural Networks Brady Neal Sarthak Mittal A. Baratin Vinayak Tantia Matthew Scicluna Simon Lacoste-Julien Ioannis Mitliagkas 26 167 0 19 Oct 2018
Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning Charles H. Martin Michael W. Mahoney AI4CE 30 190 0 02 Oct 2018
An analytic theory of generalization dynamics and transfer learning in deep linear networks Andrew Kyle Lampinen Surya Ganguli OOD 20 127 0 27 Sep 2018
The jamming transition as a paradigm to understand the loss landscape of deep neural networks Mario Geiger S. Spigler Stéphane dÁscoli Levent Sagun M. Baity-Jesi Giulio Biroli M. Wyart 11 141 0 25 Sep 2018
On the Learning Dynamics of Deep Neural Networks Rémi Tachet des Combes Mohammad Pezeshki Samira Shabanian Aaron Courville Yoshua Bengio 6 38 0 18 Sep 2018
Towards Understanding Regularization in Batch Normalization Ping Luo Xinjiang Wang Wenqi Shao Zhanglin Peng MLT AI4CE 8 179 0 04 Sep 2018
Generalization Error in Deep Learning Daniel Jakubovitz Raja Giryes M. Rodrigues AI4CE 16 109 0 03 Aug 2018
On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Length Stanislaw Jastrzebski Zachary Kenton Nicolas Ballas Asja Fischer Yoshua Bengio Amos Storkey ODL 13 115 0 13 Jul 2018
On the Spectral Bias of Neural Networks Nasim Rahaman A. Baratin Devansh Arpit Felix Dräxler Min-Bin Lin Fred Hamprecht Yoshua Bengio Aaron Courville 28 1,386 0 22 Jun 2018
Learning Dynamics of Linear Denoising Autoencoders Arnu Pretorius Steve Kroon Herman Kamper AI4CE 13 25 0 14 Jun 2018
Minnorm training: an algorithm for training over-parameterized deep neural networks Yamini Bansal Madhu S. Advani David D. Cox Andrew M. Saxe ODL 11 18 0 03 Jun 2018
The Dynamics of Learning: A Random Matrix Approach Zhenyu Liao Romain Couillet AI4CE 8 42 0 30 May 2018
Optimal ridge penalty for real-world high-dimensional data can be zero or negative due to the implicit ridge regularization D. Kobak Jonathan Lomond Benoit Sanchez 11 89 0 28 May 2018
Entropy and mutual information in models of deep neural networks Marylou Gabrié Andre Manoel Clément Luneau Jean Barbier N. Macris Florent Krzakala Lenka Zdeborová 30 178 0 24 May 2018
Deep learning generalizes because the parameter-function map is biased towards simple functions Guillermo Valle Pérez Chico Q. Camargo A. Louis MLT AI4CE 14 225 0 22 May 2018
A Study on Overfitting in Deep Reinforcement Learning Chiyuan Zhang Oriol Vinyals Rémi Munos Samy Bengio OffRL OnRL 8 383 0 18 Apr 2018
A high-bias, low-variance introduction to Machine Learning for physicists Pankaj Mehta Marin Bukov Ching-Hao Wang A. G. Day C. Richardson Charles K. Fisher D. Schwab AI4CE 11 866 0 23 Mar 2018
A Walk with SGD Chen Xing Devansh Arpit Christos Tsirigotis Yoshua Bengio 17 118 0 24 Feb 2018
Towards Understanding the Generalization Bias of Two Layer Convolutional Linear Classifiers with Gradient Descent Yifan Wu Barnabás Póczós Aarti Singh MLT 14 8 0 13 Feb 2018
Fix your classifier: the marginal value of training the last weight layer Elad Hoffer Itay Hubara Daniel Soudry 24 101 0 14 Jan 2018