Generalization error of random features and kernel methods: hypercontractivity and kernel matrix concentration

26 January 2021

Papers citing "Generalization error of random features and kernel methods: hypercontractivity and kernel matrix concentration"

42 / 92 papers shown

Title
Strong inductive biases provably prevent harmless interpolation Michael Aerni Marco Milanta Konstantin Donhauser Fanny Yang 87 9 0 18 Jan 2023
Bayesian Interpolation with Deep Linear Networks Boris Hanin Alexander Zlokapa 137 26 0 29 Dec 2022
Random Feature Models for Learning Interacting Dynamical Systems Yuxuan Liu S. McCalla Hayden Schaeffer 101 12 0 11 Dec 2022
A note on the prediction error of principal component regression in high dimensions L. Hucker Martin Wahl 64 6 0 09 Dec 2022
Dense Hebbian neural networks: a replica symmetric picture of supervised learning E. Agliari L. Albanese Francesco Alemanno Andrea Alessandrelli Adriano Barra F. Giannotti Daniele Lotito D. Pedreschi 48 18 0 25 Nov 2022
Overparameterized random feature regression with nearly orthogonal data Zhichao Wang Yizhe Zhu 75 4 0 11 Nov 2022
Learning Single-Index Models with Shallow Neural Networks A. Bietti Joan Bruna Clayton Sanford M. Song 225 71 0 27 Oct 2022
Small Transformers Compute Universal Metric Embeddings Anastasis Kratsios Valentin Debarnot Ivan Dokmanić 126 11 0 14 Sep 2022
A Universal Trade-off Between the Model Size, Test Loss, and Training Loss of Linear Predictors Nikhil Ghosh M. Belkin 74 7 0 23 Jul 2022
Identifying good directions to escape the NTK regime and efficiently learn low-degree plus sparse polynomials Eshaan Nichani Yunzhi Bai Jason D. Lee 77 10 0 08 Jun 2022
Precise Learning Curves and Higher-Order Scaling Limits for Dot Product Kernel Regression Lechao Xiao Hong Hu Theodor Misiakiewicz Yue M. Lu Jeffrey Pennington 125 20 0 30 May 2022
Excess Risk of Two-Layer ReLU Neural Networks in Teacher-Student Settings and its Superiority to Kernel Methods Shunta Akiyama Taiji Suzuki 57 6 0 30 May 2022
On the Inconsistency of Kernel Ridgeless Regression in Fixed Dimensions Daniel Beaglehole M. Belkin Parthe Pandit 62 11 0 26 May 2022
Bandwidth Selection for Gaussian Kernel Ridge Regression via Jacobian Control Oskar Allerbo Rebecka Jörnsten 59 2 0 24 May 2022
Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterization Simone Bombari Mohammad Hossein Amani Marco Mondelli 85 26 0 20 May 2022
Sharp Asymptotics of Kernel Ridge Regression Beyond the Linear Regime Hong Hu Yue M. Lu 92 16 0 13 May 2022
High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation Jimmy Ba Murat A. Erdogdu Taiji Suzuki Zhichao Wang Denny Wu Greg Yang MLT 96 129 0 03 May 2022
Spectrum of inner-product kernel matrices in the polynomial regime and multiple descent phenomenon in kernel ridge regression Theodor Misiakiewicz 54 40 0 21 Apr 2022
SRMD: Sparse Random Mode Decomposition Nicholas Richardson Hayden Schaeffer Giang Tran 46 11 0 12 Apr 2022
Adversarial Examples in Random Neural Networks with General Activations Andrea Montanari Yuchen Wu GAN AAML 103 14 0 31 Mar 2022
Deep Regression Ensembles Antoine Didisheim Bryan Kelly Semyon Malamud UQCV 46 4 0 10 Mar 2022
Failure and success of the spectral bias prediction for Kernel Ridge Regression: the case of low-dimensional data Umberto M. Tomasini Antonio Sclocchi Matthieu Wyart 70 12 0 07 Feb 2022
HARFE: Hard-Ridge Random Feature Expansion Esha Saha Hayden Schaeffer Giang Tran 119 15 0 06 Feb 2022
Eigenspace Restructuring: a Principle of Space and Frequency in Neural Networks Lechao Xiao 110 22 0 10 Dec 2021
Learning with convolution and pooling operations in kernel methods Theodor Misiakiewicz Song Mei MLT 90 29 0 16 Nov 2021
The Three Stages of Learning Dynamics in High-Dimensional Kernel Methods Nikhil Ghosh Song Mei Bin Yu 73 20 0 13 Nov 2021
Harmless interpolation in regression and classification with structured features Andrew D. McRae Santhosh Karnik Mark A. Davenport Vidya Muthukumar 184 11 0 09 Nov 2021
Conditioning of Random Feature Matrices: Double Descent and Generalization Error Zhijun Chen Hayden Schaeffer 109 12 0 21 Oct 2021
On the Double Descent of Random Features Models Trained with SGD Fanghui Liu Johan A. K. Suykens Volkan Cevher MLT 101 10 0 13 Oct 2021
Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks Zhichao Wang Yizhe Zhu 104 20 0 20 Sep 2021
Reconstruction on Trees and Low-Degree Polynomials Frederic Koehler Elchanan Mossel 70 10 0 14 Sep 2021
A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning Yehuda Dar Vidya Muthukumar Richard G. Baraniuk 117 72 0 06 Sep 2021
Deep Networks Provably Classify Data on Curves Tingran Wang Sam Buchanan D. Gilboa John N. Wright 83 9 0 29 Jul 2021
Random feature neural networks learn Black-Scholes type PDEs without curse of dimensionality Lukas Gonon 77 37 0 14 Jun 2021
How rotational invariance of common kernels prevents generalization in high dimensions Konstantin Donhauser Mingqi Wu Fanny Yang 80 24 0 09 Apr 2021
Minimum complexity interpolation in random features models Michael Celentano Theodor Misiakiewicz Andrea Montanari 31 4 0 30 Mar 2021
Exact Gap between Generalization Error and Uniform Convergence in Random Feature Models Zitong Yang Yu Bai Song Mei 71 18 0 08 Mar 2021
Learning with invariances in random features and kernel models Song Mei Theodor Misiakiewicz Andrea Montanari OOD 107 91 0 25 Feb 2021
Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeed Maria Refinetti Sebastian Goldt Florent Krzakala Lenka Zdeborová 85 74 0 23 Feb 2021
Benign overfitting in ridge regression Alexander Tsigler Peter L. Bartlett 86 169 0 29 Sep 2020
The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Training Andrea Montanari Yiqiao Zhong 187 97 0 25 Jul 2020
Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond Fanghui Liu Xiaolin Huang Yudong Chen Johan A. K. Suykens BDL 124 176 0 23 Apr 2020