v1v2v3 (latest)

Exact expressions for double descent and implicit regularization via surrogate random design

Neural Information Processing Systems (NeurIPS), 2019

10 December 2019

Papers citing "Exact expressions for double descent and implicit regularization via surrogate random design"

50 / 61 papers shown

Title
Matricial Free Energy as a Gaussianizing Regularizer: Enhancing Autoencoders for Gaussian Code Generation Rishi Sonthalia Raj Rao Nadakuditi 96 0 0 20 Oct 2025
Eigenspectrum Analysis of Neural Networks without Aspect Ratio Bias Yuanzhe Hu Kinshuk Goel Vlad Killiakov Yaoqing Yang 281 3 0 06 Jun 2025
A Model Zoo on Phase Transitions in Neural Networks Konstantin Schurholt Léo Meynent Yefan Zhou Haiquan Lu Yaoqing Yang Damian Borth 330 2 0 25 Apr 2025
Double Descent Meets Out-of-Distribution Detection: Theoretical Insights and Empirical Analysis on the role of model complexity Mouin Ben Ammar David Brellmann Arturo Mendoza Antoine Manzanera Gianni Franchi OODD 332 0 0 04 Nov 2024
Generalization for Least Squares Regression With Simple Spiked Covariances Jiping Li Rishi Sonthalia 199 1 0 17 Oct 2024
Recent and Upcoming Developments in Randomized Numerical Linear Algebra for Machine Learning Michał Dereziński Michael W. Mahoney 244 18 0 17 Jun 2024
Near-Interpolators: Rapid Norm Growth and the Trade-Off between Interpolation and GeneralizationInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2024 Yutong Wang Rishi Sonthalia Wei Hu 272 7 0 12 Mar 2024
"Lossless" Compression of Deep Neural Networks: A High-dimensional Neural Tangent Kernel Approach Lingyu Gu Yongqiang Du Yuan Zhang Di Xie Shiliang Pu Robert C. Qiu Zhenyu Liao 199 9 0 01 Mar 2024
A PAC-Bayesian Perspective on the Interpolating Information Criterion Liam Hodgkinson Christopher van der Heide Roberto Salomone Fred Roosta Michael W. Mahoney 259 2 0 13 Nov 2023
On High-Dimensional Asymptotic Properties of Model Averaging Estimators Ryo Ando F. Komaki MoMe 95 6 0 18 Aug 2023
Does Double Descent Occur in Self-Supervised Learning? A. Lupidi Yonatan Gideoni Dulhan Jayalath SSL 128 3 0 15 Jul 2023
The Interpolating Information Criterion for Overparameterized Models Liam Hodgkinson Christopher van der Heide Roberto Salomone Fred Roosta Michael W. Mahoney 190 10 0 15 Jul 2023
Least Squares Regression Can Exhibit Under-Parameterized Double DescentNeural Information Processing Systems (NeurIPS), 2023 Xinyue Li Rishi Sonthalia 290 5 0 24 May 2023
High Dimensional Binary Classification under Label Shift: Phase Transition and RegularizationSampling Theory, Signal Processing, and Data Analysis (SampTA), 2022 Jiahui Cheng Minshuo Chen Hao Liu Tuo Zhao Wenjing Liao 281 1 0 01 Dec 2022
Asymptotics of the Sketched PseudoinverseSIAM Journal on Mathematics of Data Science (SIMODS), 2022 Daniel LeJeune Pratik V. Patil Hamid Javadi Richard G. Baraniuk Robert Tibshirani 176 11 0 07 Nov 2022
Monotonicity and Double Descent in Uncertainty Estimation with Gaussian ProcessesInternational Conference on Machine Learning (ICML), 2022 Liam Hodgkinson Christopher van der Heide Fred Roosta Michael W. Mahoney UQCV 233 7 0 14 Oct 2022
Sharp Analysis of Sketch-and-Project Methods via a Connection to Randomized Singular Value DecompositionSIAM Journal on Mathematics of Data Science (SIMODS), 2022 Michal Derezinski E. Rebrova 226 23 0 20 Aug 2022
Investigating the Impact of Model Width and Density on Generalization in Presence of Label NoiseConference on Uncertainty in Artificial Intelligence (UAI), 2022 Yihao Xue Kyle Whitecross Baharan Mirzasoleiman NoLa 333 2 0 17 Aug 2022
Implicit Regularization or Implicit Conditioning? Exact Risk Trajectories of SGD in High DimensionsNeural Information Processing Systems (NeurIPS), 2022 Courtney Paquette Elliot Paquette Ben Adlam Jeffrey Pennington 113 19 0 15 Jun 2022
Hardness of Maximum Likelihood Learning of DPPsAnnual Conference Computational Learning Theory (COLT), 2022 Elena Grigorescu Brendan Juba K. Wimmer Ning Xie 148 6 0 24 May 2022
The Directional Bias Helps Stochastic Gradient Descent to Generalize in Kernel Regression ModelsInternational Symposium on Information Theory (ISIT), 2022 Yiling Luo X. Huo Y. Mei 198 0 0 29 Apr 2022
Implicit Regularization Properties of Variance Reduced Stochastic Mirror DescentInternational Symposium on Information Theory (ISIT), 2022 Yiling Luo X. Huo Y. Mei 161 5 0 29 Apr 2022
Bias-variance decomposition of overparameterized regression with random linear featuresPhysical Review E (Phys. Rev. E), 2022 J. Rocks Pankaj Mehta 115 12 0 10 Mar 2022
On Optimal Early Stopping: Over-informative versus Under-informative Parametrization Ruoqi Shen Liyao (Mars) Gao Yi-An Ma 206 16 0 20 Feb 2022
Geometric Regularization from Overparameterization Nicholas J. Teague 102 1 0 18 Feb 2022
Stochastic Perturbations of Tabular Features for Non-Deterministic Inference with Automunge Nicholas J. Teague AAML 170 1 0 18 Feb 2022
Model, sample, and epoch-wise descents: exact solution of gradient flow in the random feature modelNeural Information Processing Systems (NeurIPS), 2021 A. Bodin N. Macris 241 14 0 22 Oct 2021
On the Double Descent of Random Features Models Trained with SGD Fanghui Liu Johan A. K. Suykens Volkan Cevher MLT 415 11 0 13 Oct 2021
Random matrices in service of ML footprint: ternary random features with no performance loss Hafiz Tiomoko Ali Zhenyu Liao Romain Couillet 212 8 0 05 Oct 2021
A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning Yehuda Dar Vidya Muthukumar Richard G. Baraniuk 233 77 0 06 Sep 2021
On the Role of Optimization in Double Descent: A Least Squares StudyNeural Information Processing Systems (NeurIPS), 2021 Ilja Kuzborskij Csaba Szepesvári Omar Rivasplata Amal Rannen-Triki Razvan Pascanu 109 10 0 27 Jul 2021
Taxonomizing local versus global structure in neural network loss landscapesNeural Information Processing Systems (NeurIPS), 2021 Yaoqing Yang Liam Hodgkinson Ryan Theisen Joe Zou Joseph E. Gonzalez Kannan Ramchandran Michael W. Mahoney 355 43 0 23 Jul 2021
Towards an Understanding of Benign Overfitting in Neural Networks Zhu Li Zhi Zhou Arthur Gretton MLT 239 35 0 06 Jun 2021
The Geometry of Over-parameterized Regression and Adversarial Perturbations J. Rocks Pankaj Mehta AAML 155 9 0 25 Mar 2021
Noisy Recurrent Neural NetworksNeural Information Processing Systems (NeurIPS), 2021 Soon Hoe Lim N. Benjamin Erichson Liam Hodgkinson Michael W. Mahoney 248 65 0 09 Feb 2021
Provable Benefits of Overparameterization in Model Compression: From Double Descent to Pruning Neural NetworksAAAI Conference on Artificial Intelligence (AAAI), 2020 Xiangyu Chang Yingcong Li Samet Oymak Christos Thrampoulidis 248 58 0 16 Dec 2020
Risk-Monotonicity in Statistical LearningNeural Information Processing Systems (NeurIPS), 2020 Zakaria Mhammedi 462 8 0 28 Nov 2020
Sparse sketches with small inversion biasAnnual Conference Computational Learning Theory (COLT), 2020 Michal Derezinski Zhenyu Liao Guang Cheng Michael W. Mahoney 418 24 0 21 Nov 2020
Determinantal Point Processes Implicitly Regularize Semi-parametric Regression ProblemsSIAM Journal on Mathematics of Data Science (SIMODS), 2020 Michaël Fanuel J. Schreurs Johan A. K. Suykens 136 4 0 13 Nov 2020
Memorizing without overfitting: Bias, variance, and interpolation in over-parameterized modelsPhysical Review Research (PRResearch), 2020 J. Rocks Pankaj Mehta 385 54 0 26 Oct 2020
Failures of model-dependent generalization bounds for least-norm interpolationJournal of machine learning research (JMLR), 2020 Peter L. Bartlett Philip M. Long 351 31 0 16 Oct 2020
What causes the test error? Going beyond bias-variance via ANOVAJournal of machine learning research (JMLR), 2020 Licong Lin Guang Cheng 252 35 0 11 Oct 2020
Kernel regression in high dimensions: Refined analysis beyond double descent Fanghui Liu Zhenyu Liao Johan A. K. Suykens 288 49 0 06 Oct 2020
Benign overfitting in ridge regression Alexander Tsigler Peter L. Bartlett 271 195 0 29 Sep 2020
Provable More Data Hurt in High Dimensional Least Squares Estimator Zeng Li Chuanlong Xie Qinwen Wang 111 6 0 14 Aug 2020
Understanding Implicit Regularization in Over-Parameterized Single Index ModelJournal of the American Statistical Association (JASA), 2020 Jianqing Fan Zhuoran Yang Mengxin Yu 257 22 0 16 Jul 2020
Debiasing Distributed Second Order Optimization with Surrogate Sketching and Scaled Regularization Michal Derezinski Burak Bartan Mert Pilanci Michael W. Mahoney 155 27 0 02 Jul 2020
Ensemble Kernel Methods, Implicit Regularization and Determinantal Point Processes J. Schreurs Michaël Fanuel Johan A. K. Suykens 263 2 0 24 Jun 2020
Good Classifiers are Abundant in the Interpolating Regime Ryan Theisen Jason M. Klusowski Michael W. Mahoney 143 2 0 22 Jun 2020
Precise expressions for random projections: Low-rank approximation and randomized Newton Michal Derezinski Feynman T. Liang Zhenyu A. Liao Michael W. Mahoney 380 24 0 18 Jun 2020