Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent

8 July 2022

Tianhao Wang

Papers citing "Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent"

22 / 22 papers shown

Title
Mirror, Mirror of the Flow: How Does Regularization Shape Implicit Bias? Tom Jacobs Chao Zhou R. Burkholz OffRL AI4CE 23 0 0 17 Apr 2025
Sign-In to the Lottery: Reparameterizing Sparse Training From Scratch Advait Gadhikar Tom Jacobs Chao Zhou R. Burkholz 17 0 0 17 Apr 2025
Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy Chen Wang Kaiyi Ji Junyi Geng Zhongqiang Ren Taimeng Fu ... Yi Du Qihang Li Y. Yang Xiao Lin Zhipeng Zhao SSL 69 8 0 28 Jan 2025
Optimization Insights into Deep Diagonal Linear Networks Hippolyte Labarrière C. Molinari Lorenzo Rosasco S. Villa Cristian Vega 66 0 0 21 Dec 2024
A Mirror Descent Perspective of Smoothed Sign Descent Shuyang Wang Diego Klabjan 26 0 0 18 Oct 2024
Mask in the Mirror: Implicit Sparsification Tom Jacobs R. Burkholz 34 3 0 19 Aug 2024
Implicit Bias of Mirror Flow on Separable Data Scott Pesme Radu-Alexandru Dragomir Nicolas Flammarion 27 1 0 18 Jun 2024
Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning D. Kunin Allan Raventós Clémentine Dominé Feng Chen David Klindt Andrew M. Saxe Surya Ganguli MLT 22 15 0 10 Jun 2024
Implicit Regularization of Gradient Flow on One-Layer Softmax Attention Heejune Sheen Siyu Chen Tianhao Wang Harrison H. Zhou MLT 23 10 0 13 Mar 2024
Improving Implicit Regularization of SGD with Preconditioning for Least Square Problems Junwei Su Difan Zou Chuan Wu 14 0 0 13 Mar 2024
Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks Hristo Papazov Scott Pesme Nicolas Flammarion 22 5 0 08 Mar 2024
Achieving Margin Maximization Exponentially Fast via Progressive Norm Rescaling Mingze Wang Zeping Min Lei Wu 17 3 0 24 Nov 2023
Abide by the Law and Follow the Flow: Conservation Laws for Gradient Flows Sibylle Marcotte Rémi Gribonval Gabriel Peyré 17 9 0 30 Jun 2023
Combining Explicit and Implicit Regularization for Efficient Learning in Deep Networks Dan Zhao 14 5 0 01 Jun 2023
mSAM: Micro-Batch-Averaged Sharpness-Aware Minimization Kayhan Behdin Qingquan Song Aman Gupta S. Keerthi Ayan Acharya Borja Ocejo Gregory Dexter Rajiv Khanna D. Durfee Rahul Mazumder AAML 13 7 0 19 Feb 2023
Implicit Regularization Leads to Benign Overfitting for Sparse Linear Regression Mo Zhou Rong Ge 11 2 0 01 Feb 2023
Implicit Regularization for Group Sparsity Jiangyuan Li THANH VAN NGUYEN C. Hegde Raymond K. W. Wong 14 9 0 29 Jan 2023
Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing Jikai Jin Zhiyuan Li Kaifeng Lyu S. Du Jason D. Lee MLT 28 34 0 27 Jan 2023
Deep Linear Networks can Benignly Overfit when Shallow Ones Do Niladri S. Chatterji Philip M. Long 8 8 0 19 Sep 2022
Non-convex online learning via algorithmic equivalence Udaya Ghai Zhou Lu Elad Hazan 8 8 0 30 May 2022
Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks Noam Razin Asaf Maman Nadav Cohen 26 29 0 27 Jan 2022
What Happens after SGD Reaches Zero Loss? --A Mathematical Framework Zhiyuan Li Tianhao Wang Sanjeev Arora MLT 83 98 0 13 Oct 2021