A Mean Field View of the Landscape of Two-Layers Neural Networks

18 April 2018

Papers citing "A Mean Field View of the Landscape of Two-Layers Neural Networks"

50 / 168 papers shown

Title
Mirror Mean-Field Langevin Dynamics Anming Gu Juno Kim 29 0 0 05 May 2025
Don't be lazy: CompleteP enables compute-efficient deep transformers Nolan Dey Bin Claire Zhang Lorenzo Noci Mufan Bill Li Blake Bordelon Shane Bergsma C. Pehlevan Boris Hanin Joel Hestness 39 0 0 02 May 2025
Deep learning with missing data Tianyi Ma Tengyao Wang R. Samworth 59 0 0 21 Apr 2025
Statistically guided deep learning Michael Kohler A. Krzyżak ODL BDL 66 0 0 11 Apr 2025
Analyzing the Role of Permutation Invariance in Linear Mode Connectivity Keyao Zhan Puheng Li Lei Wu MoMe 77 0 0 13 Mar 2025
A distributional simplicity bias in the learning dynamics of transformers Riccardo Rende Federica Gerace A. Laio Sebastian Goldt 71 8 0 17 Feb 2025
A theoretical framework for overfitting in energy-based modeling Giovanni Catania A. Decelle Cyril Furtlehner Beatriz Seoane 57 2 0 31 Jan 2025
Mean-Field Analysis for Learning Subspace-Sparse Polynomials with Gaussian Input Ziang Chen Rong Ge MLT 59 1 0 10 Jan 2025
Emergence of meta-stable clustering in mean-field transformer models Giuseppe Bruno Federico Pasqualotto Andrea Agazzi 45 6 0 30 Oct 2024
Estimating the Spectral Moments of the Kernel Integral Operator from Finite Sample Matrices Chanwoo Chun SueYeon Chung Daniel D. Lee 24 1 0 23 Oct 2024
Robust Feature Learning for Multi-Index Models in High Dimensions Alireza Mousavi-Hosseini Adel Javanmard Murat A. Erdogdu OOD AAML 42 1 0 21 Oct 2024
Extended convexity and smoothness and their applications in deep learning Binchuan Qi Wei Gong Li Li 58 0 0 08 Oct 2024
The Optimization Landscape of SGD Across the Feature Learning Strength Alexander B. Atanasov Alexandru Meterez James B. Simon C. Pehlevan 43 2 0 06 Oct 2024
From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks Clémentine Dominé Nicolas Anguita A. Proca Lukas Braun D. Kunin P. Mediano Andrew M. Saxe 30 3 0 22 Sep 2024
Learning Multi-Index Models with Neural Networks via Mean-Field Langevin Dynamics Alireza Mousavi-Hosseini Denny Wu Murat A. Erdogdu MLT AI4CE 27 6 0 14 Aug 2024
Equidistribution-based training of Free Knot Splines and ReLU Neural Networks Simone Appella S. Arridge Chris Budd Teo Deveney L. Kreusser 33 0 0 02 Jul 2024
Symmetries in Overparametrized Neural Networks: A Mean-Field View Javier Maass Martínez Joaquin Fontbona FedML MLT 29 2 0 30 May 2024
Repetita Iuvant: Data Repetition Allows SGD to Learn High-Dimensional Multi-Index Functions Luca Arnaboldi Yatin Dandi Florent Krzakala Luca Pesce Ludovic Stephan 61 12 0 24 May 2024
Understanding Optimal Feature Transfer via a Fine-Grained Bias-Variance Analysis Yufan Li Subhabrata Sen Ben Adlam MLT 41 1 0 18 Apr 2024
Convergence analysis of controlled particle systems arising in deep learning: from finite to infinite sample size Huafu Liao Alpár R. Mészáros Chenchen Mou Chao Zhou 21 2 0 08 Apr 2024
$How does promoting the minority fraction affect generalization? A theoretical study of the one-hidden-layer neural network on group imbalance$ How does promoting the minority fraction affect generalization? A theoretical study of the one-hidden-layer neural network on group imbalance Hongkang Li Shuai Zhang Yihua Zhang Meng Wang Sijia Liu Pin-Yu Chen 33 4 0 12 Mar 2024
Data Reconstruction Attacks and Defenses: A Systematic Evaluation Sheng Liu Zihan Wang Yuxiao Chen Qi Lei AAML MIACV 59 4 0 13 Feb 2024
$The boundary of neural network trainability is fractal$ The boundary of neural network trainability is fractal Jascha Narain Sohl-Dickstein 24 8 0 09 Feb 2024
Mean-field underdamped Langevin dynamics and its spacetime discretization Qiang Fu Ashia Wilson 32 4 0 26 Dec 2023
Distributed Constrained Combinatorial Optimization leveraging Hypergraph Neural Networks Nasimeh Heydaribeni Xinrui Zhan Ruisi Zhang Tina Eliassi-Rad F. Koushanfar AI4CE 27 8 0 15 Nov 2023
Accelerating optimization over the space of probability measures Shi Chen Wenxuan Wu Yuhang Yao Stephen J. Wright 16 4 0 06 Oct 2023
Fundamental Limits of Deep Learning-Based Binary Classifiers Trained with Hinge Loss T. Getu Georges Kaddoum M. Bennis 32 1 0 13 Sep 2023
Gradient-Based Feature Learning under Structured Data Alireza Mousavi-Hosseini Denny Wu Taiji Suzuki Murat A. Erdogdu MLT 32 18 0 07 Sep 2023
Kernel Limit of Recurrent Neural Networks Trained on Ergodic Data Sequences Samuel Chun-Hei Lam Justin A. Sirignano K. Spiliopoulos 14 2 0 28 Aug 2023
Nonlinear Hamiltonian Monte Carlo & its Particle Approximation Nawaf Bou-Rabee Katharina Schuh 18 7 0 22 Aug 2023
Quantitative CLTs in Deep Neural Networks Stefano Favaro Boris Hanin Domenico Marinucci I. Nourdin G. Peccati BDL 23 11 0 12 Jul 2023
The RL Perceptron: Generalisation Dynamics of Policy Learning in High Dimensions Nishil Patel Sebastian Lee Stefano Sarao Mannelli Sebastian Goldt Adrew Saxe OffRL 20 3 0 17 Jun 2023
Generalization Guarantees of Gradient Descent for Multi-Layer Neural Networks Puyu Wang Yunwen Lei Di Wang Yiming Ying Ding-Xuan Zhou MLT 22 3 0 26 May 2023
High-dimensional scaling limits and fluctuations of online least-squares SGD with smooth covariance Krishnakumar Balasubramanian Promit Ghosal Ye He 28 5 0 03 Apr 2023
Doubly Regularized Entropic Wasserstein Barycenters Lénaïc Chizat 13 11 0 21 Mar 2023
Global Optimality of Elman-type RNN in the Mean-Field Regime Andrea Agazzi Jian-Xiong Lu Sayan Mukherjee MLT 20 1 0 12 Mar 2023
Phase Diagram of Initial Condensation for Two-layer Neural Networks Zheng Chen Yuqing Li Tao Luo Zhaoguang Zhou Z. Xu MLT AI4CE 41 8 0 12 Mar 2023
Critical Points and Convergence Analysis of Generative Deep Linear Networks Trained with Bures-Wasserstein Loss Pierre Bréchet Katerina Papagiannouli Jing An Guido Montúfar 18 3 0 06 Mar 2023
Primal and Dual Analysis of Entropic Fictitious Play for Finite-sum Problems Atsushi Nitanda Kazusato Oko Denny Wu Nobuhito Takenouchi Taiji Suzuki 24 3 0 06 Mar 2023
Learning time-scales in two-layers neural networks Raphael Berthier Andrea Montanari Kangjie Zhou 36 33 0 28 Feb 2023
Stochastic Modified Flows, Mean-Field Limits and Dynamics of Stochastic Gradient Descent Benjamin Gess Sebastian Kassing Vitalii Konarovskyi DiffM 24 6 0 14 Feb 2023
From high-dimensional & mean-field dynamics to dimensionless ODEs: A unifying approach to SGD in two-layers networks Luca Arnaboldi Ludovic Stephan Florent Krzakala Bruno Loureiro MLT 30 31 0 12 Feb 2023
Over-parameterised Shallow Neural Networks with Asymmetrical Node Scaling: Global Convergence Guarantees and Feature Learning François Caron Fadhel Ayed Paul Jung Hoileong Lee Juho Lee Hongseok Yang 59 2 0 02 Feb 2023
Dissecting the Effects of SGD Noise in Distinct Regimes of Deep Learning Antonio Sclocchi Mario Geiger M. Wyart 32 6 0 31 Jan 2023
Generalization on the Unseen, Logic Reasoning and Degree Curriculum Emmanuel Abbe Samy Bengio Aryo Lotfi Kevin Rizk LRM 28 47 0 30 Jan 2023
Learning Gaussian Mixtures Using the Wasserstein-Fisher-Rao Gradient Flow Yuling Yan Kaizheng Wang Philippe Rigollet 39 20 0 04 Jan 2023
An Analysis of Attention via the Lens of Exchangeability and Latent Variable Models Yufeng Zhang Boyi Liu Qi Cai Lingxiao Wang Zhaoran Wang 45 11 0 30 Dec 2022
The Underlying Correlated Dynamics in Neural Training Rotem Turjeman Tom Berkov I. Cohen Guy Gilboa 19 3 0 18 Dec 2022
Uniform-in-time propagation of chaos for mean field Langevin dynamics Fan Chen Zhenjie Ren Song-bo Wang 43 30 0 06 Dec 2022
Infinite-width limit of deep linear neural networks Lénaïc Chizat Maria Colombo Xavier Fernández-Real Alessio Figalli 31 14 0 29 Nov 2022