Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data

Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data

13 October 2022

Gal Vardi

Peter L. Bartlett

Papers citing "Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data"

11 / 11 papers shown

Title
How Transformers Learn Regular Language Recognition: A Theoretical Study on Training Dynamics and Implicit Bias Ruiquan Huang Yingbin Liang Jing Yang 43 0 0 02 May 2025
Gradient Descent Robustly Learns the Intrinsic Dimension of Data in Training Convolutional Neural Networks Chenyang Zhang Peifeng Gao Difan Zou Yuan Cao OOD MLT 59 0 0 11 Apr 2025
Neural Redshift: Random Networks are not Random Functions Damien Teney A. Nicolicioiu Valentin Hartmann Ehsan Abbasnejad 86 18 0 04 Mar 2024
Implicit Bias and Fast Convergence Rates for Self-attention Bhavya Vasudeva Puneesh Deora Christos Thrampoulidis 24 13 0 08 Feb 2024
Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization Hancheng Min Enrique Mallada René Vidal MLT 23 19 0 24 Jul 2023
Robust Sparse Mean Estimation via Incremental Learning Jianhao Ma Ruidi Chen Yinghui He S. Fattahi Wei Hu 16 0 0 24 May 2023
On the Implicit Bias in Deep-Learning Algorithms Gal Vardi FedML AI4CE 22 72 0 26 Aug 2022
On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias Itay Safran Gal Vardi Jason D. Lee MLT 41 23 0 18 May 2022
On Margin Maximization in Linear and ReLU Networks Gal Vardi Ohad Shamir Nathan Srebro 34 27 0 06 Oct 2021
Towards Understanding Learning in Neural Networks with Linear Teachers Roei Sarussi Alon Brutzkus Amir Globerson FedML MLT 44 20 0 07 Jan 2021
Provable Generalization of SGD-trained Neural Networks of Any Width in the Presence of Adversarial Label Noise Spencer Frei Yuan Cao Quanquan Gu FedML MLT 45 18 0 04 Jan 2021