The Implicit Bias of Depth: How Incremental Learning Drives Generalization

26 September 2019

Papers citing "The Implicit Bias of Depth: How Incremental Learning Drives Generalization"

21 / 21 papers shown

Title
Deep Weight Factorization: Sparse Learning Through the Lens of Artificial Symmetries Chris Kolb T. Weber Bernd Bischl David Rügamer 117 0 0 04 Feb 2025
From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks Clémentine Dominé Nicolas Anguita A. Proca Lukas Braun D. Kunin P. Mediano Andrew M. Saxe 43 3 0 22 Sep 2024
Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation Can Yaras Peng Wang Laura Balzano Qing Qu AI4CE 42 14 0 06 Jun 2024
Connectivity Shapes Implicit Regularization in Matrix Factorization Models for Matrix Completion Zhiwei Bai Jiajie Zhao Tao Luo AI4CE 37 0 0 22 May 2024
Understanding the Double Descent Phenomenon in Deep Learning Marc Lafon Alexandre Thomas 27 2 0 15 Mar 2024
Robust Sparse Mean Estimation via Incremental Learning Jianhao Ma Ruidi Chen Yinghui He S. Fattahi Wei Hu 38 0 0 24 May 2023
Gradient Descent Monotonically Decreases the Sharpness of Gradient Flow Solutions in Scalar Networks and Beyond Itai Kreisler Mor Shpigel Nacson Daniel Soudry Y. Carmon 35 13 0 22 May 2023
Robust Implicit Regularization via Weight Normalization H. Chou Holger Rauhut Rachel A. Ward 40 7 0 09 May 2023
Saddle-to-Saddle Dynamics in Diagonal Linear Networks Scott Pesme Nicolas Flammarion 44 35 0 02 Apr 2023
Learning time-scales in two-layers neural networks Raphael Berthier Andrea Montanari Kangjie Zhou 41 33 0 28 Feb 2023
Implicit Regularization Leads to Benign Overfitting for Sparse Linear Regression Mo Zhou Rong Ge 46 2 0 01 Feb 2023
Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing Jikai Jin Zhiyuan Li Kaifeng Lyu S. Du Jason D. Lee MLT 56 34 0 27 Jan 2023
Blessing of Nonconvexity in Deep Linear Models: Depth Flattens the Optimization Landscape Around the True Solution Jianhao Ma S. Fattahi 49 5 0 15 Jul 2022
Provable Acceleration of Heavy Ball beyond Quadratics for a Class of Polyak-Łojasiewicz Functions when the Non-Convexity is Averaged-Out Jun-Kun Wang Chi-Heng Lin Andre Wibisono Bin Hu 38 20 0 22 Jun 2022
What do CNNs Learn in the First Layer and Why? A Linear Systems Perspective Rhea Chowers Yair Weiss 35 2 0 06 Jun 2022
Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks Noam Razin Asaf Maman Nadav Cohen 49 29 0 27 Jan 2022
The loss landscape of deep linear neural networks: a second-order analysis El Mehdi Achour Franccois Malgouyres Sébastien Gerchinovitz ODL 26 9 0 28 Jul 2021
A Unifying View on Implicit Bias in Training Linear Neural Networks Chulhee Yun Shankar Krishnan H. Mobahi MLT 26 80 0 06 Oct 2020
Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy E. Moroshko Suriya Gunasekar Blake E. Woodworth Jason D. Lee Nathan Srebro Daniel Soudry 35 85 0 13 Jul 2020
Shape Matters: Understanding the Implicit Bias of the Noise Covariance Jeff Z. HaoChen Colin Wei Jason D. Lee Tengyu Ma 32 94 0 15 Jun 2020
Implicit Regularization in Deep Learning May Not Be Explainable by Norms Noam Razin Nadav Cohen 24 155 0 13 May 2020