Generalization error of random features and kernel methods: hypercontractivity and kernel matrix concentration

26 January 2021

Papers citing "Generalization error of random features and kernel methods: hypercontractivity and kernel matrix concentration"

50 / 92 papers shown

Title
Querying Kernel Methods Suffices for Reconstructing their Training Data Daniel Barzilai Yuval Margalit Eitan Gronich Gilad Yehudai Meirav Galun Ronen Basri 30 0 0 25 May 2025
Sobolev norm inconsistency of kernel interpolation Yunfei Yang 107 0 0 29 Apr 2025
Weak-to-Strong Generalization Even in Random Feature Networks, Provably Marko Medvedev Kaifeng Lyu Dingli Yu Sanjeev Arora Zhiyuan Li Nathan Srebro 149 3 0 04 Mar 2025
On the Saturation Effects of Spectral Algorithms in Large Dimensions Weihao Lu Haobo Zhang Yicheng Li Q. Lin 99 2 0 01 Mar 2025
Cauchy Random Features for Operator Learning in Sobolev Space Chunyang Liao Deanna Needell Hayden Schaeffer 120 1 0 01 Mar 2025
Spurious Correlations in High Dimensional Regression: The Roles of Regularization, Simplicity Bias and Over-Parameterization Simone Bombari Marco Mondelli 297 0 0 03 Feb 2025
Random Feature Models with Learnable Activation Functions Zailin Ma Jiansheng Yang Yaodong Yang 122 1 0 29 Nov 2024
A Random Matrix Theory Perspective on the Spectrum of Learned Features and Asymptotic Generalization Capabilities Yatin Dandi Luca Pesce Hugo Cui Florent Krzakala Yue M. Lu Bruno Loureiro MLT 116 2 0 24 Oct 2024
Generalization for Least Squares Regression With Simple Spiked Covariances Jiping Li Rishi Sonthalia 54 1 0 17 Oct 2024
Overfitting Behaviour of Gaussian Kernel Ridgeless Regression: Varying Bandwidth or Dimensionality Marko Medvedev Gal Vardi Nathan Srebro 94 3 0 05 Sep 2024
On the Pinsker bound of inner product kernel regression in large dimensions Weihao Lu Jialin Ding Haobo Zhang Qian Lin 87 1 0 02 Sep 2024
Operator Learning Using Random Features: A Tool for Scientific Computing Nicholas H. Nelsen Andrew M. Stuart 99 14 0 12 Aug 2024
Early learning of the optimal constant solution in neural networks and humans Jirko Rubruck Jan P. Bauer Andrew M. Saxe Christopher Summerfield 126 2 0 25 Jun 2024
Universal randomised signatures for generative time series modelling Francesca Biagini Lukas Gonon Niklas Walter 66 4 0 14 Jun 2024
High-Dimensional Kernel Methods under Covariate Shift: Data-Dependent Implicit Regularization Yihang Chen Fanghui Liu Taiji Suzuki Volkan Cevher 81 1 0 05 Jun 2024
The phase diagram of kernel interpolation in large dimensions Haobo Zhang Weihao Lu Qian Lin 69 6 0 19 Apr 2024
How Deep Networks Learn Sparse and Hierarchical Data: the Sparse Random Hierarchy Model Umberto M. Tomasini Matthieu Wyart BDL 103 7 0 16 Apr 2024
ANOVA-boosting for Random Fourier Features Daniel Potts Laura Weidensager 79 2 0 03 Apr 2024
Asymptotics of Learning with Deep Structured (Random) Features Dominik Schröder Daniil Dmitriev Hugo Cui Bruno Loureiro 80 7 0 21 Feb 2024
Asymptotics of feature learning in two-layer networks after one gradient-step Hugo Cui Luca Pesce Yatin Dandi Florent Krzakala Yue M. Lu Lenka Zdeborová Bruno Loureiro MLT 135 19 0 07 Feb 2024
Characterizing Overfitting in Kernel Ridgeless Regression Through the Eigenspectrum Tin Sum Cheng Aurelien Lucchi Anastasis Kratsios David Belius 89 8 0 02 Feb 2024
Spectrally Transformed Kernel Regression Runtian Zhai Rattana Pukdee Roger Jin Maria-Florina Balcan Pradeep Ravikumar BDL 74 2 0 01 Feb 2024
Generalization in Kernel Regression Under Realistic Assumptions Daniel Barzilai Ohad Shamir 114 17 0 26 Dec 2023
Learning from higher-order statistics, efficiently: hypothesis tests, random features, and neural networks Eszter Székely Lorenzo Bardone Federica Gerace Sebastian Goldt 73 2 0 22 Dec 2023
More is Better in Modern Machine Learning: when Infinite Overparameterization is Optimal and Overfitting is Obligatory James B. Simon Dhruva Karkada Nikhil Ghosh Mikhail Belkin AI4CE BDL 114 14 0 24 Nov 2023
Weight fluctuations in (deep) linear neural networks and a derivation of the inverse-variance flatness relation Markus Gross A. Raulf Christoph Räth 106 0 0 23 Nov 2023
A Theoretical Analysis of the Test Error of Finite-Rank Kernel Ridge Regression Tin Sum Cheng Aurelien Lucchi Ivan Dokmanić Anastasis Kratsios David Belius 61 5 0 02 Oct 2023
Fixing the NTK: From Neural Network Linearizations to Exact Convex Programs Rajat Vadiraj Dwaraknath Tolga Ergen Mert Pilanci 140 0 0 26 Sep 2023
Regret-Optimal Federated Transfer Learning for Kernel Regression with Applications in American Option Pricing Xuwei Yang Anastasis Kratsios Florian Krach Matheus Grasselli Aurelien Lucchi FedML 59 2 0 08 Sep 2023
Optimal Rate of Kernel Regression in Large Dimensions Weihao Lu Hao Zhang Yicheng Li Manyun Xu Qian Lin 86 6 0 08 Sep 2023
Six Lectures on Linearized Neural Networks Theodor Misiakiewicz Andrea Montanari 134 13 0 25 Aug 2023
Universal Approximation Theorem and error bounds for quantum neural networks and quantum reservoirs Lukas Gonon A. Jacquier 78 15 0 24 Jul 2023
What can a Single Attention Layer Learn? A Study Through the Random Features Lens Hengyu Fu Tianyu Guo Yu Bai Song Mei MLT 98 26 0 21 Jul 2023
An Agnostic View on the Cost of Overfitting in (Kernel) Ridge Regression Lijia Zhou James B. Simon Gal Vardi Nathan Srebro 57 2 0 22 Jun 2023
Nonparametric regression using over-parameterized shallow ReLU neural networks Yunfei Yang Ding-Xuan Zhou 151 6 0 14 Jun 2023
Escaping mediocrity: how two-layer networks learn hard generalized linear models with SGD Luca Arnaboldi Florent Krzakala Bruno Loureiro Ludovic Stephan MLT 99 5 0 29 May 2023
How Two-Layer Neural Networks Learn, One (Giant) Step at a Time Yatin Dandi Florent Krzakala Bruno Loureiro Luca Pesce Ludovic Stephan MLT 122 29 0 29 May 2023
Learning Capacity: A Measure of the Effective Dimensionality of a Model Daiwei Chen Wei-Di Chang Pratik Chaudhari 70 4 0 27 May 2023
Error Bounds for Learning with Vector-Valued Random Features S. Lanthaler Nicholas H. Nelsen 86 16 0 26 May 2023
How many samples are needed to leverage smoothness? Vivien A. Cabannes Stefano Vigogna 62 2 0 25 May 2023
Least Squares Regression Can Exhibit Under-Parameterized Double Descent Xinyue Li Rishi Sonthalia 109 3 0 24 May 2023
Lp- and Risk Consistency of Localized SVMs Hannes Köhler 112 0 0 16 May 2023
Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks Eshaan Nichani Alexandru Damian Jason D. Lee MLT 198 15 0 11 May 2023
New Equivalences Between Interpolation and SVMs: Kernels and Structured Features Chiraag Kaushik Andrew D. McRae Mark A. Davenport Vidya Muthukumar 90 2 0 03 May 2023
Spectral clustering in the Gaussian mixture block model Shuangping Li T. Schramm 90 8 0 29 Apr 2023
Online Learning for the Random Feature Model in the Student-Teacher Framework Roman Worschech B. Rosenow 86 0 0 24 Mar 2023
Learning time-scales in two-layers neural networks Raphael Berthier Andrea Montanari Kangjie Zhou 196 38 0 28 Feb 2023
Are Gaussian data all you need? Extents and limits of universality in high-dimensional generalized linear estimation Luca Pesce Florent Krzakala Bruno Loureiro Ludovic Stephan 84 28 0 17 Feb 2023
Deterministic equivalent and error universality of deep random features learning Dominik Schröder Hugo Cui Daniil Dmitriev Bruno Loureiro MLT 99 29 0 01 Feb 2023
Bayes-optimal Learning of Deep Random Networks of Extensive-width Hugo Cui Florent Krzakala Lenka Zdeborová BDL 103 39 0 01 Feb 2023