A Mean Field View of the Landscape of Two-Layers Neural Networks

18 April 2018

Papers citing "A Mean Field View of the Landscape of Two-Layers Neural Networks"

50 / 174 papers shown

Title
REPAIR: REnormalizing Permuted Activations for Interpolation Repair Keller Jordan Hanie Sedghi O. Saukh R. Entezari Behnam Neyshabur MoMe 46 94 0 15 Nov 2022
Stochastic Mirror Descent in Average Ensemble Models Taylan Kargin Fariborz Salehi B. Hassibi 11 1 0 27 Oct 2022
Proximal Mean Field Learning in Shallow Neural Networks Alexis M. H. Teter Iman Nodozi A. Halder FedML 35 1 0 25 Oct 2022
Global Convergence of SGD On Two Layer Neural Nets Pulkit Gopalani Anirbit Mukherjee 18 5 0 20 Oct 2022
Wasserstein Barycenter-based Model Fusion and Linear Mode Connectivity of Neural Networks A. K. Akash Sixu Li Nicolas García Trillos 24 12 0 13 Oct 2022
Annihilation of Spurious Minima in Two-Layer ReLU Networks Yossi Arjevani M. Field 16 8 0 12 Oct 2022
Meta-Principled Family of Hyperparameter Scaling Strategies Sho Yaida 50 16 0 10 Oct 2022
Analysis of the rate of convergence of an over-parametrized deep neural network estimate learned by gradient descent Michael Kohler A. Krzyżak 21 10 0 04 Oct 2022
Lazy vs hasty: linearization in deep networks impacts learning schedule based on example difficulty Thomas George Guillaume Lajoie A. Baratin 18 5 0 19 Sep 2022
Robustness in deep learning: The good (width), the bad (depth), and the ugly (initialization) Zhenyu Zhu Fanghui Liu Grigorios G. Chrysos V. Cevher 37 19 0 15 Sep 2022
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 243 313 0 11 Sep 2022
On the universal consistency of an over-parametrized deep neural network estimate learned by gradient descent Selina Drews Michael Kohler 17 13 0 30 Aug 2022
The Neural Race Reduction: Dynamics of Abstraction in Gated Networks Andrew M. Saxe Shagun Sodhani Sam Lewallen AI4CE 28 34 0 21 Jul 2022
Neural Networks can Learn Representations with Gradient Descent Alexandru Damian Jason D. Lee Mahdi Soltanolkotabi SSL MLT 14 112 0 30 Jun 2022
Learning sparse features can lead to overfitting in neural networks Leonardo Petrini Francesco Cagnetta Eric Vanden-Eijnden M. Wyart MLT 25 23 0 24 Jun 2022
Label noise (stochastic) gradient descent implicitly solves the Lasso for quadratic parametrisation Loucas Pillaud-Vivien J. Reygner Nicolas Flammarion NoLa 31 31 0 20 Jun 2022
Unbiased Estimation using Underdamped Langevin Dynamics Hamza Ruzayqat Neil K. Chada Ajay Jasra 29 4 0 14 Jun 2022
High-dimensional limit theorems for SGD: Effective dynamics and critical scaling Gerard Ben Arous Reza Gheissari Aukosh Jagannath 43 59 0 08 Jun 2022
Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks Blake Bordelon C. Pehlevan MLT 24 79 0 19 May 2022
Mean-Field Nonparametric Estimation of Interacting Particle Systems Rentian Yao Xiaohui Chen Yun Yang 43 9 0 16 May 2022
Trajectory Inference via Mean-field Langevin in Path Space Lénaïc Chizat Stephen X. Zhang Matthieu Heitz Geoffrey Schiebinger 18 20 0 14 May 2022
High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation Jimmy Ba Murat A. Erdogdu Taiji Suzuki Zhichao Wang Denny Wu Greg Yang MLT 29 121 0 03 May 2022
On the (Non-)Robustness of Two-Layer Neural Networks in Different Learning Regimes Elvis Dohmatob A. Bietti AAML 21 13 0 22 Mar 2022
Fully-Connected Network on Noncompact Symmetric Space and Ridgelet Transform based on Helgason-Fourier Analysis Sho Sonoda Isao Ishikawa Masahiro Ikeda 13 15 0 03 Mar 2022
A blob method for inhomogeneous diffusion with applications to multi-agent control and sampling Katy Craig Karthik Elamvazhuthi M. Haberland O. Turanova 25 15 0 25 Feb 2022
Provably convergent quasistatic dynamics for mean-field two-player zero-sum games Chao Ma Lexing Ying MLT 19 11 0 15 Feb 2022
Random Feature Amplification: Feature Learning and Generalization in Neural Networks Spencer Frei Niladri S. Chatterji Peter L. Bartlett MLT 30 29 0 15 Feb 2022
Simultaneous Transport Evolution for Minimax Equilibria on Measures Carles Domingo-Enrich Joan Bruna 6 3 0 14 Feb 2022
Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks R. Veiga Ludovic Stephan Bruno Loureiro Florent Krzakala Lenka Zdeborová MLT 10 31 0 01 Feb 2022
Improved Overparametrization Bounds for Global Convergence of Stochastic Gradient Descent for Shallow Neural Networks Bartlomiej Polaczyk J. Cyranka ODL 28 3 0 28 Jan 2022
Convex Analysis of the Mean Field Langevin Dynamics Atsushi Nitanda Denny Wu Taiji Suzuki MLT 59 64 0 25 Jan 2022
Overview frequency principle/spectral bias in deep learning Z. Xu Yaoyu Zhang Tao Luo FaML 25 65 0 19 Jan 2022
Convergence of Policy Gradient for Entropy Regularized MDPs with Neural Network Approximation in the Mean-Field Regime B. Kerimkulov J. Leahy David Siska Lukasz Szpruch 22 11 0 18 Jan 2022
Neural Capacitance: A New Perspective of Neural Network Selection via Edge Dynamics Chunheng Jiang Tejaswini Pedapati Pin-Yu Chen Yizhou Sun Jianxi Gao 16 2 0 11 Jan 2022
Separation of Scales and a Thermodynamic Description of Feature Learning in Some CNNs Inbar Seroussi Gadi Naveh Z. Ringel 25 49 0 31 Dec 2021
DNN gradient lossless compression: Can GenNorm be the answer? Zhongzhu Chen Eduin E. Hernandez Yu-Chih Huang Stefano Rini 9 9 0 15 Nov 2021
Efficient Neural Network Training via Forward and Backward Propagation Sparsification Xiao Zhou Weizhong Zhang Zonghao Chen Shizhe Diao Tong Zhang 24 46 0 10 Nov 2021
Mean-field Analysis of Piecewise Linear Solutions for Wide ReLU Networks A. Shevchenko Vyacheslav Kungurtsev Marco Mondelli MLT 25 13 0 03 Nov 2021
Sinkformers: Transformers with Doubly Stochastic Attention Michael E. Sander Pierre Ablin Mathieu Blondel Gabriel Peyré 18 76 0 22 Oct 2021
The Convex Geometry of Backpropagation: Neural Network Gradient Flows Converge to Extreme Points of the Dual Convex Program Yifei Wang Mert Pilanci MLT MDE 47 11 0 13 Oct 2021
Parallel Deep Neural Networks Have Zero Duality Gap Yifei Wang Tolga Ergen Mert Pilanci 71 10 0 13 Oct 2021
The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks R. Entezari Hanie Sedghi O. Saukh Behnam Neyshabur MoMe 35 215 0 12 Oct 2021
AIR-Net: Adaptive and Implicit Regularization Neural Network for Matrix Completion Zhemin Li Tao Sun Hongxia Wang Bao Wang 42 6 0 12 Oct 2021
Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networks Shuai Zhang Meng Wang Sijia Liu Pin-Yu Chen Jinjun Xiong UQCV MLT 13 13 0 12 Oct 2021
Tighter Sparse Approximation Bounds for ReLU Neural Networks Carles Domingo-Enrich Youssef Mroueh 89 4 0 07 Oct 2021
On the Global Convergence of Gradient Descent for multi-layer ResNets in the mean-field regime Zhiyan Ding Shi Chen Qin Li S. Wright MLT AI4CE 30 11 0 06 Oct 2021
The emergence of a concept in shallow neural networks E. Agliari Francesco Alemanno Adriano Barra G. D. Marzo 18 39 0 01 Sep 2021
The loss landscape of deep linear neural networks: a second-order analysis E. M. Achour Franccois Malgouyres Sébastien Gerchinovitz ODL 22 9 0 28 Jul 2021
Analytic Study of Families of Spurious Minima in Two-Layer ReLU Neural Networks: A Tale of Symmetry II Yossi Arjevani M. Field 28 18 0 21 Jul 2021
The Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusion D. Kunin Javier Sagastuy-Breña Lauren Gillespie Eshed Margalit Hidenori Tanaka Surya Ganguli Daniel L. K. Yamins 28 15 0 19 Jul 2021