Early Neuron Alignment in Two-layer ReLU Networks with Small
Initialization

Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization

24 July 2023

Enrique Mallada

Papers citing "Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization"

14 / 14 papers shown

Title
An overview of condensation phenomenon in deep learning Zhi-Qin John Xu Yaoyu Zhang Zhangchen Zhou AI4CE 24 0 0 13 Apr 2025
Understanding the Learning Dynamics of LoRA: A Gradient Flow Perspective on Low-Rank Adaptation in Matrix Factorization Ziqing Xu Hancheng Min Lachlan Ewen MacDonald Jinqi Luo Salma Tarmoun Enrique Mallada René Vidal AI4CE 49 0 0 10 Mar 2025
Low-rank bias, weight decay, and model merging in neural networks Ilja Kuzborskij Yasin Abbasi-Yadkori 47 0 0 24 Feb 2025
Simplicity bias and optimization threshold in two-layer ReLU networks Etienne Boursier Nicolas Flammarion 24 2 0 03 Oct 2024
TSVD: Bridging Theory and Practice in Continual Learning with Pre-trained Models Liangzu Peng Juan Elenter Joshua Agterberg Alejandro Ribeiro René Vidal VLM CLL 44 1 0 01 Oct 2024
Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning D. Kunin Allan Raventós Clémentine Dominé Feng Chen David Klindt Andrew M. Saxe Surya Ganguli MLT 22 15 0 10 Jun 2024
Simplicity Bias of Two-Layer Networks beyond Linearly Separable Data Nikita Tsoy Nikola Konstantinov 21 4 0 27 May 2024
Can Implicit Bias Imply Adversarial Robustness? Hancheng Min René Vidal 24 3 0 24 May 2024
Early Directional Convergence in Deep Homogeneous Neural Networks for Small Initializations Akshay Kumar Jarvis D. Haupt ODL 42 3 0 12 Mar 2024
On the Convergence of Differentially-Private Fine-tuning: To Linearly Probe or to Fully Fine-tune? Shuqi Ke Charlie Hou Giulia Fanti Sewoong Oh 23 4 0 29 Feb 2024
Directional Convergence Near Small Initializations and Saddles in Two-Homogeneous Neural Networks Akshay Kumar Jarvis D. Haupt ODL 19 6 0 14 Feb 2024
Understanding Unimodal Bias in Multimodal Deep Linear Networks Yedi Zhang Peter E. Latham Andrew Saxe 15 5 0 01 Dec 2023
Learning a Neuron by a Shallow ReLU Network: Dynamics and Implicit Bias for Correlated Inputs D. Chistikov Matthias Englert R. Lazic MLT 30 12 0 10 Jun 2023
Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks Noam Razin Asaf Maman Nadav Cohen 28 29 0 27 Jan 2022