The Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of Generalization

15 August 2020

Papers citing "The Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of Generalization"

50 / 94 papers shown

Title
Eigenvalue distribution of the Neural Tangent Kernel in the quadratic scaling Lucas Benigni Elliot Paquette 40 0 0 27 Aug 2025
Geometry of Neural Reinforcement Learning in Continuous State and Action Spaces Saket Tiwari Omer Gottesman George Konidaris 82 0 0 28 Jul 2025
Eigenspectrum Analysis of Neural Networks without Aspect Ratio Bias Yuanzhe Hu Kinshuk Goel Vlad Killiakov Yaoqing Yang 153 2 0 06 Jun 2025
Random at First, Fast at Last: NTK-Guided Fourier Pre-Processing for Tabular DL Renat Sergazinov Jing Wu Shao-An Yin AAML LMTD 110 0 0 03 Jun 2025
auto-fpt: Automating Free Probability Theory Calculations for Machine Learning Theory Arjun Subramonian Elvis Dohmatob 123 0 0 14 Apr 2025
Gradient Descent Robustly Learns the Intrinsic Dimension of Data in Training Convolutional Neural Networks Chenyang Zhang Peifeng Gao Difan Zou Yuan Cao OOD MLT 197 0 0 11 Apr 2025
High-dimensional ridge regression with random features for non-identically distributed data with a variance profile Issa-Mbenard Dabo Jérémie Bigot 89 0 0 03 Apr 2025
Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer Blake Bordelon Cengiz Pehlevan AI4CE 427 4 0 04 Feb 2025
Generalization for Least Squares Regression With Simple Spiked Covariances Jiping Li Rishi Sonthalia 119 1 0 17 Oct 2024
Strong Model Collapse Elvis Dohmatob Yunzhen Feng Arjun Subramonian Julia Kempe 128 25 0 07 Oct 2024
Implicit Regularization Paths of Weighted Neural Representations Jin-Hong Du Pratik Patil 115 0 0 28 Aug 2024
From Spikes to Heavy Tails: Unveiling the Spectral Evolution of Neural Networks Vignesh Kothapalli Tianyu Pang Shenyang Deng Zongmin Liu Yaoqing Yang 138 4 0 07 Jun 2024
Learning Analysis of Kernel Ridgeless Regression with Asymmetric Kernel Learning Fan He Mingzhe He Lei Shi Xiaolin Huang Johan A. K. Suykens 109 1 0 03 Jun 2024
Signal-Plus-Noise Decomposition of Nonlinear Spiked Random Matrix Models Behrad Moniri Hamed Hassani 151 2 0 28 May 2024
High dimensional analysis reveals conservative sharpening and a stochastic edge of stability Atish Agarwala Jeffrey Pennington 171 7 0 30 Apr 2024
Understanding Optimal Feature Transfer via a Fine-Grained Bias-Variance Analysis Yufan Li Subhabrata Sen Ben Adlam MLT 221 2 0 18 Apr 2024
Theoretical and Empirical Insights into the Origins of Degree Bias in Graph Neural Networks Arjun Subramonian Jian Kang Yizhou Sun AI4CE 99 6 0 04 Apr 2024
Benign overfitting in leaky ReLU networks with moderate input dimension Kedar Karhadkar Erin E. George Michael Murray Guido Montúfar Deanna Needell MLT 152 2 0 11 Mar 2024
"Lossless" Compression of Deep Neural Networks: A High-dimensional Neural Tangent Kernel Approach Lingyu Gu Yongqiang Du Yuan Zhang Di Xie Shiliang Pu Robert C. Qiu Zhenyu Liao 119 9 0 01 Mar 2024
Asymptotics of Learning with Deep Structured (Random) Features Dominik Schröder Daniil Dmitriev Hugo Cui Bruno Loureiro 153 8 0 21 Feb 2024
Average gradient outer product as a mechanism for deep neural collapse Daniel Beaglehole Peter Súkeník Marco Mondelli Misha Belkin AI4CE 164 15 0 21 Feb 2024
Feature learning as alignment: a structural property of gradient descent in non-linear neural networks Daniel Beaglehole Ioannis Mitliagkas Atish Agarwala MLT 143 3 0 07 Feb 2024
A Dynamical Model of Neural Scaling Laws Blake Bordelon Alexander B. Atanasov Cengiz Pehlevan 281 52 0 02 Feb 2024
The Surprising Harmfulness of Benign Overfitting for Adversarial Robustness Yifan Hao Tong Zhang AAML 219 5 0 19 Jan 2024
A Theory of Non-Linear Feature Learning with One Gradient Step in Two-Layer Neural Networks Behrad Moniri Donghwan Lee Hamed Hassani Guang Cheng MLT 212 29 0 11 Oct 2023
Asymptotically free sketched ridge ensembles: Risks, cross-validation, and tuning Filip Szatkowski Daniel LeJeune 141 10 0 06 Oct 2023
Corrected generalized cross-validation for finite ensembles of penalized estimators Pierre C. Bellec Jin-Hong Du Takuya Koriyama Pratik Patil Kai Tan 190 5 0 02 Oct 2023
Fundamental Limits of Deep Learning-Based Binary Classifiers Trained with Hinge Loss T. Getu Georges Kaddoum M. Bennis 155 1 0 13 Sep 2023
Six Lectures on Linearized Neural Networks Theodor Misiakiewicz Andrea Montanari 208 15 0 25 Aug 2023
Eight challenges in developing theory of intelligence Haiping Huang 127 12 0 20 Jun 2023
Training shallow ReLU networks on noisy data using hinge loss: when do we overfit and is it benign? Erin E. George Michael Murray W. Swartworth Deanna Needell MLT 106 5 0 16 Jun 2023
Asymptotics of Bayesian Uncertainty Estimation in Random Features Regression You-Hyun Baek S. Berchuck Sayan Mukherjee 171 0 0 06 Jun 2023
Neural (Tangent Kernel) Collapse Mariia Seleznova Dana Weitzner Raja Giryes Gitta Kutyniok H. Chou 152 12 0 25 May 2023
Least Squares Regression Can Exhibit Under-Parameterized Double Descent Xinyue Li Rishi Sonthalia 146 4 0 24 May 2023
How Spurious Features Are Memorized: Precise Analysis for Random and NTK Features Simone Bombari Marco Mondelli AAML 147 9 0 20 May 2023
Subsample Ridge Ensembles: Equivalences and Generalized Cross-Validation Jin-Hong Du Pratik V. Patil Arun K. Kuchibhotla 125 11 0 25 Apr 2023
Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networks Blake Bordelon Cengiz Pehlevan MLT 151 36 0 06 Apr 2023
Gradient flow on extensive-rank positive semi-definite matrix denoising A. Bodin N. Macris 111 4 0 16 Mar 2023
Benign Overfitting for Two-layer ReLU Convolutional Neural Networks Yiwen Kou Zi-Yuan Chen Yuanzhou Chen Quanquan Gu MLT 119 18 0 07 Mar 2023
Learning curves for deep structured Gaussian feature models Jacob A. Zavatone-Veth Cengiz Pehlevan MLT 133 12 0 01 Mar 2023
Some Fundamental Aspects about Lipschitz Continuity of Neural Networks Grigory Khromov Sidak Pal Singh 252 16 0 21 Feb 2023
Beyond the Universal Law of Robustness: Sharper Laws for Random Features and Neural Tangent Kernels Simone Bombari Shayan Kiyani Marco Mondelli AAML 207 11 0 03 Feb 2023
Demystifying Disagreement-on-the-Line in High Dimensions Dong-Hwan Lee Behrad Moniri Xinmeng Huang Guang Cheng Hamed Hassani 131 12 0 31 Jan 2023
Deep networks for system identification: a Survey G. Pillonetto Aleksandr Aravkin Daniel Gedon L. Ljung Antônio H. Ribeiro Thomas B. Schon OOD 147 63 0 30 Jan 2023
Bayesian Interpolation with Deep Linear Networks Boris Hanin Alexander Zlokapa 203 28 0 29 Dec 2022
The Onset of Variance-Limited Behavior for Networks in the Lazy and Rich Regimes Alexander B. Atanasov Blake Bordelon Sabarish Sainathan Cengiz Pehlevan 150 29 0 23 Dec 2022
Gradient flow in the gaussian covariate model: exact solution of learning curves and multiple descent structures Antione Bodin N. Macris 130 4 0 13 Dec 2022
Spectral Evolution and Invariance in Linear-width Neural Networks Zhichao Wang A. Engel Anand D. Sarwate Ioana Dumitriu Tony Chiang 178 22 0 11 Nov 2022
Overparameterized random feature regression with nearly orthogonal data Zhichao Wang Yizhe Zhu 155 5 0 11 Nov 2022
Monotonicity and Double Descent in Uncertainty Estimation with Gaussian Processes Liam Hodgkinson Christopher van der Heide Fred Roosta Michael W. Mahoney UQCV 148 6 0 14 Oct 2022