High-dimensional dynamics of generalization error in neural networks

10 October 2017

Papers citing "High-dimensional dynamics of generalization error in neural networks"

50 / 296 papers shown

Title
On the geometry of generalization and memorization in deep neural networks Cory Stephenson Suchismita Padhy Abhinav Ganesh Yue Hui Hanlin Tang SueYeon Chung TDI AI4CE 11 73 0 30 May 2021
A Theory of Neural Tangent Kernel Alignment and Its Influence on Training H. Shan Blake Bordelon 11 11 0 29 May 2021
Towards Understanding the Condensation of Neural Networks at Initial Training Hanxu Zhou Qixuan Zhou Tao Luo Yaoyu Zhang Z. Xu MLT AI4CE 14 25 0 25 May 2021
Relative stability toward diffeomorphisms indicates performance in deep nets Leonardo Petrini Alessandro Favero Mario Geiger M. Wyart OOD 28 15 0 06 May 2021
The Geometry of Over-parameterized Regression and Adversarial Perturbations J. Rocks Pankaj Mehta AAML 11 7 0 25 Mar 2021
The Shape of Learning Curves: a Review T. Viering Marco Loog 18 122 0 19 Mar 2021
The Low-Rank Simplicity Bias in Deep Networks Minyoung Huh H. Mobahi Richard Y. Zhang Brian Cheung Pulkit Agrawal Phillip Isola 17 109 0 18 Mar 2021
On the interplay between data structure and loss function in classification problems Stéphane dÁscoli Marylou Gabrié Levent Sagun Giulio Biroli 21 17 0 09 Mar 2021
On the Generalization Power of Overfitted Two-Layer Neural Tangent Kernel Models Peizhong Ju Xiaojun Lin Ness B. Shroff MLT 19 9 0 09 Mar 2021
Asymptotics of Ridge Regression in Convolutional Models Mojtaba Sahraee-Ardakan Tung Mai Anup B. Rao Ryan Rossi S. Rangan A. Fletcher MLT 11 2 0 08 Mar 2021
Exact Gap between Generalization Error and Uniform Convergence in Random Feature Models Zitong Yang Yu Bai Song Mei 8 17 0 08 Mar 2021
Asymptotic Risk of Overparameterized Likelihood Models: Double Descent Theory for Deep Neural Networks Ryumei Nakada Masaaki Imaizumi 19 2 0 28 Feb 2021
Two-way kernel matrix puncturing: towards resource-efficient PCA and spectral clustering Romain Couillet Florent Chatelain N. L. Bihan 11 8 0 24 Feb 2021
Implicit Regularization in Tensor Factorization Noam Razin Asaf Maman Nadav Cohen 17 48 0 19 Feb 2021
Double-descent curves in neural networks: a new perspective using Gaussian processes Ouns El Harzli Bernardo Cuenca Grau Guillermo Valle Pérez A. Louis 13 6 0 14 Feb 2021
Learning by Turning: Neural Architecture Aware Optimisation Yang Liu Jeremy Bernstein M. Meister Yisong Yue ODL 39 26 0 14 Feb 2021
Distilling Double Descent Andrew Cotter A. Menon Harikrishna Narasimhan A. S. Rawat Sashank J. Reddi Yichen Zhou 12 7 0 13 Feb 2021
Explaining Neural Scaling Laws Yasaman Bahri Ethan Dyer Jared Kaplan Jaehoon Lee Utkarsh Sharma 19 249 0 12 Feb 2021
Meta-learning with negative learning rates A. Bernacchia 15 17 0 01 Feb 2021
A Statistician Teaches Deep Learning G. Babu David L. Banks Hyunsoo Cho David Han Hailin Sang Shouyi Wang 10 2 0 29 Jan 2021
Self-Adaptive Training: Bridging Supervised and Self-Supervised Learning Lang Huang Chaoning Zhang Hongyang R. Zhang SSL 22 24 0 21 Jan 2021
On Data-Augmentation and Consistency-Based Semi-Supervised Learning Atin Ghosh Alexandre Hoang Thiery 63 20 0 18 Jan 2021
Phases of learning dynamics in artificial neural networks: with or without mislabeled data Yu Feng Y. Tu 17 2 0 16 Jan 2021
Perspective: A Phase Diagram for Deep Learning unifying Jamming, Feature Learning and Lazy Training Mario Geiger Leonardo Petrini M. Wyart DRL 18 11 0 30 Dec 2020
Analysis of the Scalability of a Deep-Learning Network for Steganography "Into the Wild" Hugo Ruiz Marc Chaumont Mehdi Yedroudj A. Amara Frédéric Comby Gérard Subsol 13 9 0 29 Dec 2020
Avoiding The Double Descent Phenomenon of Random Feature Models Using Hybrid Regularization Kelvin K. Kan J. Nagy Lars Ruthotto AI4CE 29 6 0 11 Dec 2020
Statistical Mechanics of Deep Linear Neural Networks: The Back-Propagating Kernel Renormalization Qianyi Li H. Sompolinsky 11 69 0 07 Dec 2020
Align, then memorise: the dynamics of learning with feedback alignment Maria Refinetti Stéphane dÁscoli Ruben Ohana Sebastian Goldt 26 36 0 24 Nov 2020
Gradient Starvation: A Learning Proclivity in Neural Networks Mohammad Pezeshki Sekouba Kaba Yoshua Bengio Aaron Courville Doina Precup Guillaume Lajoie MLT 45 257 0 18 Nov 2020
Understanding Double Descent Requires a Fine-Grained Bias-Variance Decomposition Ben Adlam Jeffrey Pennington UD 24 92 0 04 Nov 2020
Memorizing without overfitting: Bias, variance, and interpolation in over-parameterized models J. Rocks Pankaj Mehta 8 41 0 26 Oct 2020
A Dynamical View on Optimization Algorithms of Overparameterized Neural Networks Zhiqi Bu Shiyun Xu Kan Chen 17 17 0 25 Oct 2020
What causes the test error? Going beyond bias-variance via ANOVA Licong Lin Edgar Dobriban 14 34 0 11 Oct 2020
On the Universality of the Double Descent Peak in Ridgeless Regression David Holzmüller 6 12 0 05 Oct 2020
Small Data, Big Decisions: Model Selection in the Small-Data Regime J. Bornschein Francesco Visin Simon Osindero 8 36 0 26 Sep 2020
Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't E. Weinan Chao Ma Stephan Wojtowytsch Lei Wu AI4CE 11 133 0 22 Sep 2020
Distributional Generalization: A New Kind of Generalization Preetum Nakkiran Yamini Bansal OOD 16 41 0 17 Sep 2020
Asymptotics of Wide Convolutional Neural Networks Anders Andreassen Ethan Dyer 12 22 0 19 Aug 2020
The Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of Generalization Ben Adlam Jeffrey Pennington 10 123 0 15 Aug 2020
Provable More Data Hurt in High Dimensional Least Squares Estimator Zeng Li Chuanlong Xie Qinwen Wang 15 6 0 14 Aug 2020
The Slow Deterioration of the Generalization Error of the Random Feature Model Chao Ma Lei Wu E. Weinan 12 15 0 13 Aug 2020
Shallow Univariate ReLu Networks as Splines: Initialization, Loss Surface, Hessian, & Gradient Flow Dynamics Justin Sahs Ryan Pyle Aneel Damaraju J. O. Caro Onur Tavaslioglu Andy Lu Ankit B. Patel 8 19 0 04 Aug 2020
Multiple Descent: Design Your Own Generalization Curve Lin Chen Yifei Min M. Belkin Amin Karbasi DRL 18 61 0 03 Aug 2020
Finite Versus Infinite Neural Networks: an Empirical Study Jaehoon Lee S. Schoenholz Jeffrey Pennington Ben Adlam Lechao Xiao Roman Novak Jascha Narain Sohl-Dickstein 17 207 0 31 Jul 2020
Geometric compression of invariant manifolds in neural nets J. Paccolat Leonardo Petrini Mario Geiger Kevin Tyloo M. Wyart MLT 49 34 0 22 Jul 2020
Early Stopping in Deep Networks: Double Descent and How to Eliminate it Reinhard Heckel Fatih Yilmaz 18 43 0 20 Jul 2020
Data-driven effective model shows a liquid-like deep learning Wenxuan Zou Haiping Huang 16 2 0 16 Jul 2020
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding Dmitry Lepikhin HyoukJoong Lee Yuanzhong Xu Dehao Chen Orhan Firat Yanping Huang M. Krikun Noam M. Shazeer Z. Chen MoE 20 1,106 0 30 Jun 2020
Statistical Mechanical Analysis of Neural Network Pruning Rupam Acharyya Ankani Chattoraj Boyu Zhang Shouman Das Daniel Stefankovic 18 0 0 30 Jun 2020
The Gaussian equivalence of generative models for learning with shallow neural networks Sebastian Goldt Bruno Loureiro Galen Reeves Florent Krzakala M. Mézard Lenka Zdeborová BDL 33 100 0 25 Jun 2020