Neural networks trained with SGD learn distributions of increasing
complexity

Neural networks trained with SGD learn distributions of increasing complexity

21 November 2022

Maria Refinetti

Alessandro Ingrosso

Sebastian Goldt

Papers citing "Neural networks trained with SGD learn distributions of increasing complexity"

11 / 11 papers shown

Title
A distributional simplicity bias in the learning dynamics of transformers Riccardo Rende Federica Gerace A. Laio Sebastian Goldt 68 8 0 17 Feb 2025
The Optimization Landscape of SGD Across the Feature Learning Strength Alexander B. Atanasov Alexandru Meterez James B. Simon C. Pehlevan 43 2 0 06 Oct 2024
How Feature Learning Can Improve Neural Scaling Laws Blake Bordelon Alexander B. Atanasov C. Pehlevan 49 12 0 26 Sep 2024
Dynamics of Supervised and Reinforcement Learning in the Non-Linear Perceptron Christian Schmid James M. Murray 32 0 0 05 Sep 2024
Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences Nikolaos Dimitriadis Pascal Frossard F. Fleuret MoE 57 6 0 10 Jul 2024
Do deep neural networks have an inbuilt Occam's razor? Chris Mingard Henry Rees Guillermo Valle Pérez A. Louis UQCV BDL 19 15 0 13 Apr 2023
Data-driven emergence of convolutional structure in neural networks Alessandro Ingrosso Sebastian Goldt 48 38 0 01 Feb 2022
The Intrinsic Dimension of Images and Its Impact on Learning Phillip E. Pope Chen Zhu Ahmed Abdelkader Micah Goldblum Tom Goldstein 183 256 0 18 Apr 2021
Pre-training without Natural Images Hirokatsu Kataoka Kazushige Okayasu Asato Matsumoto Eisuke Yamagata Ryosuke Yamada Nakamasa Inoue Akio Nakamura Y. Satoh 79 115 0 21 Jan 2021
Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networks Blake Bordelon Abdulkadir Canatar C. Pehlevan 131 199 0 07 Feb 2020
Densely Connected Convolutional Networks Gao Huang Zhuang Liu L. V. D. van der Maaten Kilian Q. Weinberger PINN 3DV 244 35,884 0 25 Aug 2016