High-dimensional dynamics of generalization error in neural networks

10 October 2017

Papers citing "High-dimensional dynamics of generalization error in neural networks"

50 / 296 papers shown

Title
The Quenching-Activation Behavior of the Gradient Descent Dynamics for Two-layer Neural Network Models Chao Ma Lei Wu E. Weinan MLT 18 10 0 25 Jun 2020
Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networks Abdulkadir Canatar Blake Bordelon C. Pehlevan 6 180 0 23 Jun 2020
An analytic theory of shallow networks dynamics for hinge loss classification Franco Pellegrini Giulio Biroli 17 19 0 19 Jun 2020
When Does Preconditioning Help or Hurt Generalization? S. Amari Jimmy Ba Roger C. Grosse Xuechen Li Atsushi Nitanda Taiji Suzuki Denny Wu Ji Xu 31 32 0 18 Jun 2020
On Sparsity in Overparametrised Shallow ReLU Networks Jaume de Dios Joan Bruna 6 14 0 18 Jun 2020
Revisiting minimum description length complexity in overparameterized models Raaz Dwivedi Chandan Singh Bin Yu Martin J. Wainwright 6 4 0 17 Jun 2020
How isotropic kernels perform on simple invariants J. Paccolat S. Spigler M. Wyart 14 4 0 17 Jun 2020
Asymptotics of Ridge (less) Regression under General Source Condition Dominic Richards Jaouad Mourtada Lorenzo Rosasco 18 72 0 11 Jun 2020
On Uniform Convergence and Low-Norm Interpolation Learning Lijia Zhou Danica J. Sutherland Nathan Srebro 14 29 0 10 Jun 2020
$On the Optimal Weighted $\ell_2$ Regularization in Overparameterized Linear Regression$ On the Optimal Weighted $\ell_2$ Regularization in Overparameterized Linear Regression Denny Wu Ji Xu 17 121 0 10 Jun 2020
A Random Matrix Analysis of Random Fourier Features: Beyond the Gaussian Kernel, a Precise Phase Transition, and the Corresponding Double Descent Zhenyu Liao Romain Couillet Michael W. Mahoney 11 87 0 09 Jun 2020
What needles do sparse neural networks find in nonlinear haystacks S. Sardy N. Hengartner Nikolai Bobenko Yen Ting Lin 14 1 0 07 Jun 2020
Triple descent and the two kinds of overfitting: Where & why do they appear? Stéphane dÁscoli Levent Sagun Giulio Biroli 8 80 0 05 Jun 2020
Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networks Z. Fan Zhichao Wang 29 72 0 25 May 2020
Implicit Regularization in Deep Learning May Not Be Explainable by Norms Noam Razin Nadav Cohen 16 155 0 13 May 2020
An Investigation of Why Overparameterization Exacerbates Spurious Correlations Shiori Sagawa Aditi Raghunathan Pang Wei Koh Percy Liang 146 370 0 09 May 2020
Generalization Error of Generalized Linear Models in High Dimensions M Motavali Emami Mojtaba Sahraee-Ardakan Parthe Pandit S. Rangan A. Fletcher AI4CE 6 37 0 01 May 2020
The Information Bottleneck Problem and Its Applications in Machine Learning Ziv Goldfeld Yury Polyanskiy 13 129 0 30 Apr 2020
Towards a theory of machine learning V. Vanchurin 11 24 0 15 Apr 2020
On the interplay between physical and content priors in deep learning for computational imaging Mo Deng Shuai Li Iksung Kang N. Fang George Barbastathis 18 26 0 14 Apr 2020
Regularization in High-Dimensional Regression and Classification via Random Matrix Theory Panagiotis Lolas 18 13 0 30 Mar 2020
Going in circles is the way forward: the role of recurrence in visual inference R. S. V. Bergen N. Kriegeskorte 17 81 0 26 Mar 2020
$On the robustness of the minimum $\ell_2$ interpolator$ On the robustness of the minimum $\ell_2$ interpolator Geoffrey Chinot M. Lerasle 20 10 0 12 Mar 2020
Correlated Initialization for Correlated Data Johannes Schneider 6 5 0 09 Mar 2020
Optimal Regularization Can Mitigate Double Descent Preetum Nakkiran Prayaag Venkat Sham Kakade Tengyu Ma 15 128 0 04 Mar 2020
Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime Stéphane dÁscoli Maria Refinetti Giulio Biroli Florent Krzakala 90 152 0 02 Mar 2020
Rethinking Bias-Variance Trade-off for Generalization of Neural Networks Zitong Yang Yaodong Yu Chong You Jacob Steinhardt Yi-An Ma 4 181 0 26 Feb 2020
Self-Adaptive Training: beyond Empirical Risk Minimization Lang Huang Chaoning Zhang Hongyang R. Zhang NoLa 21 197 0 24 Feb 2020
Generalisation error in learning with random features and the hidden manifold model Federica Gerace Bruno Loureiro Florent Krzakala M. Mézard Lenka Zdeborová 25 165 0 21 Feb 2020
Implicit Regularization of Random Feature Models Arthur Jacot Berfin Simsek Francesco Spadaro Clément Hongler Franck Gabriel 18 82 0 19 Feb 2020
Asymptotic errors for convex penalized linear regression beyond Gaussian matrices Cédric Gerbelot A. Abbara Florent Krzakala 13 16 0 11 Feb 2020
Overfitting Can Be Harmless for Basis Pursuit, But Only to a Degree Peizhong Ju Xiaojun Lin Jia Liu 72 7 0 02 Feb 2020
Analytic Study of Double Descent in Binary Classification: The Impact of Loss Ganesh Ramachandra Kini Christos Thrampoulidis 7 52 0 30 Jan 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 228 4,460 0 23 Jan 2020
Understanding Why Neural Networks Generalize Well Through GSNR of Parameters Jinlong Liu Guo-qing Jiang Yunzhi Bai Ting Chen Huayan Wang AI4CE 15 48 0 21 Jan 2020
Provable Benefit of Orthogonal Initialization in Optimizing Deep Linear Networks Wei Hu Lechao Xiao Jeffrey Pennington 19 113 0 16 Jan 2020
How neural networks find generalizable solutions: Self-tuned annealing in deep learning Yu Feng Y. Tu MLT 12 9 0 06 Jan 2020
On the Bias-Variance Tradeoff: Textbooks Need an Update Brady Neal 13 18 0 17 Dec 2019
More Data Can Hurt for Linear Regression: Sample-wise Double Descent Preetum Nakkiran 12 68 0 16 Dec 2019
Double descent in the condition number T. Poggio Gil Kur Andy Banburski 9 26 0 12 Dec 2019
Frivolous Units: Wider Networks Are Not Really That Wide Stephen Casper Xavier Boix Vanessa D’Amario Ling Guo Martin Schrimpf Kasper Vinken Gabriel Kreiman 13 19 0 10 Dec 2019
In Defense of Uniform Convergence: Generalization via derandomization with an application to interpolating predictors Jeffrey Negrea Gintare Karolina Dziugaite Daniel M. Roy AI4CE 19 64 0 09 Dec 2019
Rademacher complexity and spin glasses: A link between the replica and statistical theories of learning A. Abbara Benjamin Aubin Florent Krzakala Lenka Zdeborová 14 13 0 05 Dec 2019
Deep Double Descent: Where Bigger Models and More Data Hurt Preetum Nakkiran Gal Kaplun Yamini Bansal Tristan Yang Boaz Barak Ilya Sutskever 15 914 0 04 Dec 2019
Towards Understanding the Spectral Bias of Deep Learning Yuan Cao Zhiying Fang Yue Wu Ding-Xuan Zhou Quanquan Gu 21 214 0 03 Dec 2019
A Random Matrix Perspective on Mixtures of Nonlinearities for Deep Learning Ben Adlam J. Levinson Jeffrey Pennington 17 23 0 02 Dec 2019
On the Heavy-Tailed Theory of Stochastic Gradient Descent for Deep Neural Networks Umut Simsekli Mert Gurbuzbalaban T. H. Nguyen G. Richard Levent Sagun 13 55 0 29 Nov 2019
Information Bottleneck Theory on Convolutional Neural Networks Jianing Li Ding Liu FAtt 25 3 0 09 Nov 2019
Mean-field inference methods for neural networks Marylou Gabrié AI4CE 16 32 0 03 Nov 2019
Generalization in multitask deep neural classifiers: a statistical physics approach Tyler Lee A. Ndirango AI4CE 16 20 0 30 Oct 2019