Learning One-hidden-layer ReLU Networks via Gradient Descent

20 June 2018

Quanquan Gu

Papers citing "Learning One-hidden-layer ReLU Networks via Gradient Descent"

50 / 86 papers shown

Title
On the Hardness of Learning One Hidden Layer Neural Networks Shuchen Li Ilias Zadik Manolis Zampetakis 16 2 0 04 Oct 2024
$Iterative thresholding for non-linear learning in the strong $\varepsilon$-contamination model$ Iterative thresholding for non-linear learning in the strong $\varepsilon$ -contamination model Arvind Rathnashyam Alex Gittens 31 0 0 05 Sep 2024
Stochastic Bandits with ReLU Neural Networks Kan Xu Hamsa Bastani Surbhi Goel Osbert Bastani 27 0 0 12 May 2024
$How does promoting the minority fraction affect generalization? A theoretical study of the one-hidden-layer neural network on group imbalance$ How does promoting the minority fraction affect generalization? A theoretical study of the one-hidden-layer neural network on group imbalance Hongkang Li Shuai Zhang Yihua Zhang Meng Wang Sijia Liu Pin-Yu Chen 33 4 0 12 Mar 2024
Towards Robust Out-of-Distribution Generalization Bounds via Sharpness Yingtian Zou Kenji Kawaguchi Yingnan Liu Jiashuo Liu M. Lee W. Hsu 37 5 0 11 Mar 2024
Understanding Representation Learnability of Nonlinear Self-Supervised Learning Ruofeng Yang Xiangyuan Li Bo Jiang Shuai Li SSL MLT 45 2 0 06 Jan 2024
Improving the Expressive Power of Deep Neural Networks through Integral Activation Transform Zezhong Zhang Feng Bao Guannan Zhang 6 0 0 19 Dec 2023
Convergence Analysis for Learning Orthonormal Deep Linear Neural Networks Zhen Qin Xuwei Tan Zhihui Zhu 32 0 0 24 Nov 2023
Max-affine regression via first-order methods Seonho Kim Kiryung Lee 17 2 0 15 Aug 2023
A faster and simpler algorithm for learning shallow networks Sitan Chen Shyam Narayanan 33 7 0 24 Jul 2023
Dissecting Chain-of-Thought: Compositionality through In-Context Filtering and Learning Yingcong Li Kartik K. Sreenivasan Angeliki Giannou Dimitris Papailiopoulos Samet Oymak LRM 16 16 0 30 May 2023
Most Neural Networks Are Almost Learnable Amit Daniely Nathan Srebro Gal Vardi 13 0 0 25 May 2023
$Toward $L_\infty$-recovery of Nonlinear Functions: A Polynomial Sample Complexity Bound for Gaussian Random Fields$ Toward $L_\infty$ -recovery of Nonlinear Functions: A Polynomial Sample Complexity Bound for Gaussian Random Fields Kefan Dong Tengyu Ma 27 4 0 29 Apr 2023
Learning Narrow One-Hidden-Layer ReLU Networks Sitan Chen Zehao Dou Surbhi Goel Adam R. Klivans Raghu Meka MLT 11 13 0 20 Apr 2023
DiffFit: Unlocking Transferability of Large Diffusion Models via Simple Parameter-Efficient Fine-Tuning Enze Xie Lewei Yao Han Shi Zhili Liu Daquan Zhou Zhaoqiang Liu Jiawei Li Zhenguo Li 24 76 0 13 Apr 2023
Over-Parameterization Exponentially Slows Down Gradient Descent for Learning a Single Neuron Weihang Xu S. Du 26 16 0 20 Feb 2023
Continuized Acceleration for Quasar Convex Functions in Non-Convex Optimization Jun-Kun Wang Andre Wibisono 23 9 0 15 Feb 2023
$Generalization Ability of Wide Neural Networks on $\mathbb{R}$$ Generalization Ability of Wide Neural Networks on $\mathbb{R}$ Jianfa Lai Manyun Xu Rui Chen Qi-Rong Lin 11 21 0 12 Feb 2023
ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradients Guihong Li Yuedong Yang Kartikeya Bhardwaj R. Marculescu 28 60 0 26 Jan 2023
An Analysis of Attention via the Lens of Exchangeability and Latent Variable Models Yufeng Zhang Boyi Liu Qi Cai Lingxiao Wang Zhaoran Wang 45 11 0 30 Dec 2022
Finite Sample Identification of Wide Shallow Neural Networks with Biases M. Fornasier T. Klock Marco Mondelli Michael Rauchensteiner 17 6 0 08 Nov 2022
Neural Networks Efficiently Learn Low-Dimensional Representations with SGD Alireza Mousavi-Hosseini Sejun Park M. Girotti Ioannis Mitliagkas Murat A. Erdogdu MLT 319 48 0 29 Sep 2022
Magnitude and Angle Dynamics in Training Single ReLU Neurons Sangmin Lee Byeongsu Sim Jong Chul Ye MLT 94 6 0 27 Sep 2022
Local Identifiability of Deep ReLU Neural Networks: the Theory Joachim Bona-Pellissier Franccois Malgouyres F. Bachoc FAtt 60 6 0 15 Jun 2022
Excess Risk of Two-Layer ReLU Neural Networks in Teacher-Student Settings and its Superiority to Kernel Methods Shunta Akiyama Taiji Suzuki 22 6 0 30 May 2022
Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis Wuyang Chen Wei Huang Xinyu Gong Boris Hanin Zhangyang Wang 22 7 0 11 May 2022
Explicitising The Implicit Intrepretability of Deep Neural Networks Via Duality Chandrashekar Lakshminarayanan Ashutosh Kumar Singh A. Rajkumar AI4CE 8 1 0 01 Mar 2022
Benign Overfitting in Two-layer Convolutional Neural Networks Yuan Cao Zixiang Chen M. Belkin Quanquan Gu MLT 9 82 0 14 Feb 2022
Hardness of Noise-Free Learning for Two-Hidden-Layer Neural Networks Sitan Chen Aravind Gollakota Adam R. Klivans Raghu Meka 14 30 0 10 Feb 2022
How does unlabeled data improve generalization in self-training? A one-hidden-layer theoretical analysis Shuai Zhang M. Wang Sijia Liu Pin-Yu Chen Jinjun Xiong SSL MLT 36 22 0 21 Jan 2022
Parameter identifiability of a deep feedforward ReLU neural network Joachim Bona-Pellissier François Bachoc François Malgouyres 33 14 0 24 Dec 2021
Efficiently Learning Any One Hidden Layer ReLU Network From Queries Sitan Chen Adam R. Klivans Raghu Meka MLAU MLT 37 8 0 08 Nov 2021
Coordinate Descent Methods for DC Minimization: Optimality Conditions and Global Convergence Ganzhao Yuan 11 3 0 09 Sep 2021
How much pre-training is enough to discover a good subnetwork? Cameron R. Wolfe Fangshuo Liao Qihan Wang J. Kim Anastasios Kyrillidis 22 3 0 31 Jul 2021
Structured Directional Pruning via Perturbation Orthogonal Projection YinchuanLi XiaofengLiu YunfengShao QingWang YanhuiGeng 13 2 0 12 Jul 2021
On Learnability via Gradient Method for Two-Layer ReLU Neural Networks in Teacher-Student Setting Shunta Akiyama Taiji Suzuki MLT 11 13 0 11 Jun 2021
Self-Regularity of Non-Negative Output Weights for Overparameterized Two-Layer Neural Networks D. Gamarnik Eren C. Kizildaug Ilias Zadik 12 1 0 02 Mar 2021
Spurious Local Minima Are Common for Deep Neural Networks with Piecewise Linear Activations Bo Liu 9 7 0 25 Feb 2021
A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Network Mo Zhou Rong Ge Chi Jin 69 44 0 04 Feb 2021
Stable Recovery of Entangled Weights: Towards Robust Identification of Deep Neural Networks from Minimal Samples Christian Fiedler M. Fornasier T. Klock Michael Rauchensteiner OOD 22 11 0 18 Jan 2021
Towards Searching Efficient and Accurate Neural Network Architectures in Binary Classification Problems Yigit Can Alparslan E. Moyer I. Isozaki Daniel Ethan Schwartz Adam Dunlop Shesh Dave Edward J. Kim MQ AI4CE 15 6 0 16 Jan 2021
A Convergence Theory Towards Practical Over-parameterized Deep Neural Networks Asaf Noy Yi Tian Xu Y. Aflalo Lihi Zelnik-Manor R. L. Jin 23 3 0 12 Jan 2021
Neural Network Training Techniques Regularize Optimization Trajectory: An Empirical Study Cheng Chen Junjie Yang Yi Zhou 11 0 0 13 Nov 2020
Estimating Stochastic Linear Combination of Non-linear Regressions Efficiently and Scalably Di Wang Xiangyu Guo Chaowen Guan Shi Li Jinhui Xu 16 1 0 19 Oct 2020
Agnostic Learning of Halfspaces with Gradient Descent via Soft Margins Spencer Frei Yuan Cao Quanquan Gu 14 13 0 01 Oct 2020
Learning Deep ReLU Networks Is Fixed-Parameter Tractable Sitan Chen Adam R. Klivans Raghu Meka 14 36 0 28 Sep 2020
Nonparametric Learning of Two-Layer ReLU Residual Units Zhunxuan Wang Linyun He Chunchuan Lyu Shay B. Cohen MLT OffRL 22 1 0 17 Aug 2020
Learning Over-Parametrized Two-Layer ReLU Neural Networks beyond NTK Yuanzhi Li Tengyu Ma Hongyang R. Zhang MLT 20 28 0 09 Jul 2020
Fast Learning of Graph Neural Networks with Guaranteed Generalizability: One-hidden-layer Case Shuai Zhang Meng Wang Sijia Liu Pin-Yu Chen Jinjun Xiong MLT AI4CE 17 33 0 25 Jun 2020
Superpolynomial Lower Bounds for Learning One-Layer Neural Networks using Gradient Descent Surbhi Goel Aravind Gollakota Zhihan Jin Sushrut Karmalkar Adam R. Klivans MLT ODL 16 70 0 22 Jun 2020