Subquadratic Overparameterization for Shallow Neural Networks

2 November 2021

Papers citing "Subquadratic Overparameterization for Shallow Neural Networks"

25 / 25 papers shown

Title
MLPs at the EOC: Dynamics of Feature Learning Dávid Terjék MLT 41 0 0 18 Feb 2025
Loss Landscape Characterization of Neural Networks without Over-Parametrization Rustem Islamov Niccolò Ajroldi Antonio Orvieto Aurélien Lucchi 33 4 0 16 Oct 2024
Optimal Hessian/Jacobian-Free Nonconvex-PL Bilevel Optimization Feihu Huang 35 4 0 25 Jul 2024
Approximation and Gradient Descent Training with Neural Networks G. Welper 26 1 0 19 May 2024
Adaptive Mirror Descent Bilevel Optimization Feihu Huang 28 1 0 08 Nov 2023
On the Convergence of Encoder-only Shallow Transformers Yongtao Wu Fanghui Liu Grigorios G. Chrysos V. Cevher 42 5 0 02 Nov 2023
Global Convergence of SGD For Logistic Loss on Two Layer Neural Nets Pulkit Gopalani Samyak Jha Anirbit Mukherjee 14 2 0 17 Sep 2023
Approximation Results for Gradient Descent trained Neural Networks G. Welper 37 0 0 09 Sep 2023
On Penalty Methods for Nonconvex Bilevel Optimization and First-Order Stochastic Approximation Jeongyeol Kwon Dohyun Kwon Steve Wright Robert D. Nowak 26 25 0 04 Sep 2023
Mildly Overparameterized ReLU Networks Have a Favorable Loss Landscape Kedar Karhadkar Michael Murray Hanna Tseran Guido Montúfar 10 6 0 31 May 2023
On Momentum-Based Gradient Methods for Bilevel Optimization with Nonconvex Lower-Level Feihu Huang 19 18 0 07 Mar 2023
Learning Lipschitz Functions by GD-trained Shallow Overparameterized ReLU Neural Networks Ilja Kuzborskij Csaba Szepesvári 21 4 0 28 Dec 2022
Improved Convergence Guarantees for Shallow Neural Networks A. Razborov ODL 17 1 0 05 Dec 2022
Finite Sample Identification of Wide Shallow Neural Networks with Biases M. Fornasier T. Klock Marco Mondelli Michael Rauchensteiner 11 6 0 08 Nov 2022
Optimization for Amortized Inverse Problems Tianci Liu Tong Yang Quan Zhang Qi Lei 24 4 0 25 Oct 2022
Global Convergence of SGD On Two Layer Neural Nets Pulkit Gopalani Anirbit Mukherjee 18 5 0 20 Oct 2022
BOME! Bilevel Optimization Made Easy: A Simple First-Order Approach Mao Ye B. Liu S. Wright Peter Stone Qian Liu 72 82 0 19 Sep 2022
Approximation results for Gradient Descent trained Shallow Neural Networks in $1d$ R. Gentile G. Welper ODL 25 6 0 17 Sep 2022
Robustness in deep learning: The good (width), the bad (depth), and the ugly (initialization) Zhenyu Zhu Fanghui Liu Grigorios G. Chrysos V. Cevher 35 19 0 15 Sep 2022
Informed Learning by Wide Neural Networks: Convergence, Generalization and Sampling Complexity Jianyi Yang Shaolei Ren 13 3 0 02 Jul 2022
A Framework for Overparameterized Learning Dávid Terjék Diego González-Sánchez MLT 11 1 0 26 May 2022
Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterization Simone Bombari Mohammad Hossein Amani Marco Mondelli 12 26 0 20 May 2022
Geometric Regularization from Overparameterization Nicholas J. Teague 17 1 0 18 Feb 2022
On the Convergence of Shallow Neural Network Training with Randomly Masked Neurons Fangshuo Liao Anastasios Kyrillidis 22 16 0 05 Dec 2021
How much pre-training is enough to discover a good subnetwork? Cameron R. Wolfe Fangshuo Liao Qihan Wang J. Kim Anastasios Kyrillidis 17 3 0 31 Jul 2021