Limitations of Lazy Training of Two-layers Neural Networks

21 June 2019

Papers citing "Limitations of Lazy Training of Two-layers Neural Networks"

28 / 28 papers shown

Title
Learning Multi-Index Models with Neural Networks via Mean-Field Langevin Dynamics Alireza Mousavi-Hosseini Denny Wu Murat A. Erdogdu MLT AI4CE 27 6 0 14 Aug 2024
Disentangling and Mitigating the Impact of Task Similarity for Continual Learning Naoki Hiratani CLL 35 2 0 30 May 2024
Gradient-Based Feature Learning under Structured Data Alireza Mousavi-Hosseini Denny Wu Taiji Suzuki Murat A. Erdogdu MLT 32 18 0 07 Sep 2023
Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks Eshaan Nichani Alexandru Damian Jason D. Lee MLT 33 13 0 11 May 2023
Online Learning for the Random Feature Model in the Student-Teacher Framework Roman Worschech B. Rosenow 31 0 0 24 Mar 2023
Global Optimality of Elman-type RNN in the Mean-Field Regime Andrea Agazzi Jian-Xiong Lu Sayan Mukherjee MLT 11 1 0 12 Mar 2023
Primal and Dual Analysis of Entropic Fictitious Play for Finite-sum Problems Atsushi Nitanda Kazusato Oko Denny Wu Nobuhito Takenouchi Taiji Suzuki 24 3 0 06 Mar 2023
Generalization on the Unseen, Logic Reasoning and Degree Curriculum Emmanuel Abbe Samy Bengio Aryo Lotfi Kevin Rizk LRM 21 47 0 30 Jan 2023
Neural Networks can Learn Representations with Gradient Descent Alexandru Damian Jason D. Lee Mahdi Soltanolkotabi SSL MLT 4 112 0 30 Jun 2022
Learning sparse features can lead to overfitting in neural networks Leonardo Petrini Francesco Cagnetta Eric Vanden-Eijnden M. Wyart MLT 25 23 0 24 Jun 2022
High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation Jimmy Ba Murat A. Erdogdu Taiji Suzuki Zhichao Wang Denny Wu Greg Yang MLT 27 121 0 03 May 2022
On the (Non-)Robustness of Two-Layer Neural Networks in Different Learning Regimes Elvis Dohmatob A. Bietti AAML 15 13 0 22 Mar 2022
Random Feature Amplification: Feature Learning and Generalization in Neural Networks Spencer Frei Niladri S. Chatterji Peter L. Bartlett MLT 22 29 0 15 Feb 2022
Convex Analysis of the Mean Field Langevin Dynamics Atsushi Nitanda Denny Wu Taiji Suzuki MLT 59 64 0 25 Jan 2022
Subquadratic Overparameterization for Shallow Neural Networks Chaehwan Song Ali Ramezani-Kebrya Thomas Pethick Armin Eftekhari V. Cevher 19 31 0 02 Nov 2021
The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective Geoff Pleiss John P. Cunningham 13 24 0 11 Jun 2021
Relative stability toward diffeomorphisms indicates performance in deep nets Leonardo Petrini Alessandro Favero Mario Geiger M. Wyart OOD 13 15 0 06 May 2021
A Priori Generalization Analysis of the Deep Ritz Method for Solving High Dimensional Elliptic Equations Jianfeng Lu Yulong Lu Min Wang 15 37 0 05 Jan 2021
Align, then memorise: the dynamics of learning with feedback alignment Maria Refinetti Stéphane dÁscoli Ruben Ohana Sebastian Goldt 13 36 0 24 Nov 2020
When Does Preconditioning Help or Hurt Generalization? S. Amari Jimmy Ba Roger C. Grosse Xuechen Li Atsushi Nitanda Taiji Suzuki Denny Wu Ji Xu 24 32 0 18 Jun 2020
Shape Matters: Understanding the Implicit Bias of the Noise Covariance Jeff Z. HaoChen Colin Wei J. Lee Tengyu Ma 18 93 0 15 Jun 2020
Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond Fanghui Liu Xiaolin Huang Yudong Chen Johan A. K. Suykens BDL 22 172 0 23 Apr 2020
A Mean-field Analysis of Deep ResNet and Beyond: Towards Provable Optimization Via Overparameterization From Depth Yiping Lu Chao Ma Yulong Lu Jianfeng Lu Lexing Ying MLT 24 78 0 11 Mar 2020
Learning Parities with Neural Networks Amit Daniely Eran Malach 13 76 0 18 Feb 2020
Proving the Lottery Ticket Hypothesis: Pruning is All You Need Eran Malach Gilad Yehudai Shai Shalev-Shwartz Ohad Shamir 9 271 0 03 Feb 2020
Asymptotics of Wide Networks from Feynman Diagrams Ethan Dyer Guy Gur-Ari 17 113 0 25 Sep 2019
Linearized two-layers neural networks in high dimension Behrooz Ghorbani Song Mei Theodor Misiakiewicz Andrea Montanari MLT 11 241 0 27 Apr 2019
Sharp analysis of low-rank kernel matrix approximations Francis R. Bach 75 278 0 09 Aug 2012