v1v2 (latest)

Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias

26 October 2021

Papers citing "Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias"

26 / 26 papers shown

Title
Alternating Gradient Flows: A Theory of Feature Learning in Two-layer Neural Networks D. Kunin Giovanni Luca Marchetti F. Chen Dhruva Karkada James B. Simon M. DeWeese Surya Ganguli Nina Miolane 23 0 0 06 Jun 2025
Multiclass Loss Geometry Matters for Generalization of Gradient Descent in Separable Classification Matan Schliserman Tomer Koren 20 0 0 28 May 2025
Directional Convergence, Benign Overfitting of Gradient Descent in leaky ReLU two-layer Neural Networks Ichiro Hashimoto MLT 77 0 0 22 May 2025
Learning Guarantee of Reward Modeling Using Deep Neural Networks Yuanhang Luo Yeheng Ge Ruijian Han Guohao Shen 69 0 0 10 May 2025
Gradient Descent Robustly Learns the Intrinsic Dimension of Data in Training Convolutional Neural Networks Chenyang Zhang Peifeng Gao Difan Zou Yuan Cao OOD MLT 157 0 0 11 Apr 2025
Simplicity bias and optimization threshold in two-layer ReLU networks Etienne Boursier Nicolas Flammarion 93 4 0 03 Oct 2024
Early Directional Convergence in Deep Homogeneous Neural Networks for Small Initializations Akshay Kumar Jarvis Haupt ODL 103 4 0 12 Mar 2024
Neural Redshift: Random Networks are not Random Functions Damien Teney A. Nicolicioiu Valentin Hartmann Ehsan Abbasnejad 187 25 0 04 Mar 2024
Deep ReLU Networks Have Surprisingly Simple Polytopes Fenglei Fan Wei Huang Xiang-yu Zhong Lecheng Ruan T. Zeng Huan Xiong Fei Wang 108 5 0 16 May 2023
On the Effect of Initialization: The Scaling Path of 2-Layer Neural Networks Sebastian Neumayer Lénaïc Chizat M. Unser 65 2 0 31 Mar 2023
Lower Generalization Bounds for GD and SGD in Smooth Stochastic Convex Optimization Peiyuan Zhang Jiaye Teng J.N. Zhang 71 4 0 19 Mar 2023
Phase Diagram of Initial Condensation for Two-layer Neural Networks Zheng Chen Yuqing Li Yaoyu Zhang Zhaoguang Zhou Z. Xu MLT AI4CE 104 11 0 12 Mar 2023
Simplicity Bias in 1-Hidden Layer Neural Networks Depen Morwani Jatin Batra Prateek Jain Praneeth Netrapalli 100 21 0 01 Feb 2023
Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models Hong Liu Sang Michael Xie Zhiyuan Li Tengyu Ma AI4CE 126 55 0 25 Oct 2022
The Asymmetric Maximum Margin Bias of Quasi-Homogeneous Neural Networks D. Kunin Atsushi Yamamura Chao Ma Surya Ganguli 79 21 0 07 Oct 2022
On the Implicit Bias in Deep-Learning Algorithms Gal Vardi FedML AI4CE 80 81 0 26 Aug 2022
Feature selection with gradient descent on two-layer networks in low-rotation regimes Matus Telgarsky MLT 81 16 0 04 Aug 2022
Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent Zhiyuan Li Tianhao Wang Jason D. Lee Sanjeev Arora 100 29 0 08 Jul 2022
Max-Margin Works while Large Margin Fails: Generalization without Uniform Convergence Margalit Glasgow Colin Wei Mary Wootters Tengyu Ma 94 5 0 16 Jun 2022
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction Kaifeng Lyu Zhiyuan Li Sanjeev Arora FAtt 121 75 0 14 Jun 2022
Adversarial Reprogramming Revisited Matthias Englert R. Lazic AAML 86 11 0 07 Jun 2022
Empirical Phase Diagram for Three-layer Neural Networks with Infinite Width Hanxu Zhou Qixuan Zhou Zhenyuan Jin Yaoyu Zhang Yaoyu Zhang Zhi-Qin John Xu 55 22 0 24 May 2022
Random Feature Amplification: Feature Learning and Generalization in Neural Networks Spencer Frei Niladri S. Chatterji Peter L. Bartlett MLT 103 30 0 15 Feb 2022
Implicit Regularization Towards Rank Minimization in ReLU Networks Nadav Timor Gal Vardi Ohad Shamir 88 51 0 30 Jan 2022
Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks Noam Razin Asaf Maman Nadav Cohen 132 29 0 27 Jan 2022
On Margin Maximization in Linear and ReLU Networks Gal Vardi Ohad Shamir Nathan Srebro 162 30 0 06 Oct 2021