Bottleneck Structure in Learned Features: Low-Dimension vs Regularity
Tradeoff

v1v2 (latest)

Bottleneck Structure in Learned Features: Low-Dimension vs Regularity Tradeoff

Neural Information Processing Systems (NeurIPS), 2023

30 May 2023

ArXiv (abs)PDF HTML

Papers citing "Bottleneck Structure in Learned Features: Low-Dimension vs Regularity Tradeoff"

9 / 9 papers shown

Title
Saddle-To-Saddle Dynamics in Deep ReLU Networks: Low-Rank Bias in the First Saddle Escape Ioannis Bantzis James B. Simon Arthur Jacot ODL 300 1 0 27 May 2025
Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural CollapseInternational Conference on Learning Representations (ICLR), 2024 Arthur Jacot Peter Súkeník Zihan Wang Marco Mondelli 272 9 0 07 Oct 2024
How DNNs break the Curse of Dimensionality: Compositionality and Symmetry Learning Arthur Jacot Seok Hoan Choi Yuxiao Wen AI4CE 311 5 0 08 Jul 2024
Hamiltonian Mechanics of Feature Learning: Bottleneck Structure in Leaky ResNets Arthur Jacot Alexandre Kaiser 225 1 0 27 May 2024
Depth Separation in Norm-Bounded Infinite-Width Neural Networks Suzanna Parkinson Greg Ongie Rebecca Willett Ohad Shamir Nathan Srebro MDE 197 2 0 13 Feb 2024
Which Frequencies do CNNs Need? Emergent Bottleneck Structure in Feature LearningInternational Conference on Machine Learning (ICML), 2024 Yuxiao Wen Arthur Jacot 356 9 0 12 Feb 2024
Layer-wise Linear Mode ConnectivityInternational Conference on Learning Representations (ICLR), 2023 Linara Adilova Maksym Andriushchenko Michael Kamp Asja Fischer Martin Jaggi FedML FAtt MoMe 429 20 0 13 Jul 2023
$Implicit bias of SGD in $L_{2}$-regularized linear DNNs: One-way jumps from high to low rank$ Implicit bias of SGD in $L_{2}$ -regularized linear DNNs: One-way jumps from high to low rankInternational Conference on Learning Representations (ICLR), 2023 Zihan Wang Arthur Jacot 228 23 0 25 May 2023
ReLU Neural Networks with Linear Layers are Biased Towards Single- and Multi-Index ModelsSIAM Journal on Mathematics of Data Science (SIMODS), 2023 Suzanna Parkinson Greg Ongie Rebecca Willett 391 7 0 24 May 2023