Theoretical insights into the optimization landscape of over-parameterized shallow neural networks

16 July 2017

Papers citing "Theoretical insights into the optimization landscape of over-parameterized shallow neural networks"

50 / 76 papers shown

Title
Information-Theoretic Guarantees for Recovering Low-Rank Tensors from Symmetric Rank-One Measurements Eren C. Kızıldağ 48 0 0 07 Feb 2025
Geometry and Optimization of Shallow Polynomial Networks Yossi Arjevani Joan Bruna Joe Kileel Elzbieta Polak Matthew Trager 34 1 0 10 Jan 2025
Near-Optimal Solutions of Constrained Learning Problems Juan Elenter Luiz F. O. Chamon Alejandro Ribeiro 26 5 0 18 Mar 2024
Early Directional Convergence in Deep Homogeneous Neural Networks for Small Initializations Akshay Kumar Jarvis D. Haupt ODL 44 3 0 12 Mar 2024
Loss Landscape of Shallow ReLU-like Neural Networks: Stationary Points, Saddle Escape, and Network Embedding Zhengqing Wu Berfin Simsek Francois Ged ODL 38 0 0 08 Feb 2024
Fundamental Limits of Deep Learning-Based Binary Classifiers Trained with Hinge Loss T. Getu Georges Kaddoum M. Bennis 32 1 0 13 Sep 2023
Solving Large-scale Spatial Problems with Convolutional Neural Networks Damian Owerko Charilaos I. Kanatsoulis Alejandro Ribeiro 19 2 0 14 Jun 2023
Resilient Constrained Learning Ignacio Hounie Alejandro Ribeiro Luiz F. O. Chamon 21 9 0 04 Jun 2023
How Spurious Features Are Memorized: Precise Analysis for Random and NTK Features Simone Bombari Marco Mondelli AAML 19 4 0 20 May 2023
Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks Eshaan Nichani Alexandru Damian Jason D. Lee MLT 36 13 0 11 May 2023
NTK-SAP: Improving neural network pruning by aligning training dynamics Yite Wang Dawei Li Ruoyu Sun 28 19 0 06 Apr 2023
Beyond the Universal Law of Robustness: Sharper Laws for Random Features and Neural Tangent Kernels Simone Bombari Shayan Kiyani Marco Mondelli AAML 18 10 0 03 Feb 2023
Implementation of Trained Factorization Machine Recommendation System on Quantum Annealer Chen-Yu Liu Hsin-Yu Wang Pei-Yen Liao Ching-Jui Lai Min-hsiu Hsieh LRM 21 14 0 24 Oct 2022
When Expressivity Meets Trainability: Fewer than $n$ Neurons Can Work Jiawei Zhang Yushun Zhang Mingyi Hong Ruoyu Sun Z. Luo 19 10 0 21 Oct 2022
Annihilation of Spurious Minima in Two-Layer ReLU Networks Yossi Arjevani M. Field 16 8 0 12 Oct 2022
Intersection of Parallels as an Early Stopping Criterion Ali Vardasbi Maarten de Rijke Mostafa Dehghani MoMe 28 5 0 19 Aug 2022
Neural Networks can Learn Representations with Gradient Descent Alexandru Damian Jason D. Lee Mahdi Soltanolkotabi SSL MLT 14 112 0 30 Jun 2022
Tensor-on-Tensor Regression: Riemannian Optimization, Over-parameterization, Statistical-computational Gap, and Their Interplay Yuetian Luo Anru R. Zhang 21 19 0 17 Jun 2022
Identifying good directions to escape the NTK regime and efficiently learn low-degree plus sparse polynomials Eshaan Nichani Yunzhi Bai Jason D. Lee 8 10 0 08 Jun 2022
Non-convex online learning via algorithmic equivalence Udaya Ghai Zhou Lu Elad Hazan 8 8 0 30 May 2022
Embedding Principle in Depth for the Loss Landscape Analysis of Deep Neural Networks Zhiwei Bai Tao Luo Z. Xu Yaoyu Zhang 23 4 0 26 May 2022
On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias Itay Safran Gal Vardi Jason D. Lee MLT 44 23 0 18 May 2022
Knowledge Distillation Meets Open-Set Semi-Supervised Learning Jing Yang Xiatian Zhu Adrian Bulat Brais Martínez Georgios Tzimiropoulos 29 7 0 13 May 2022
Convergence of gradient descent for deep neural networks S. Chatterjee ODL 11 20 0 30 Mar 2022
On the (Non-)Robustness of Two-Layer Neural Networks in Different Learning Regimes Elvis Dohmatob A. Bietti AAML 21 13 0 22 Mar 2022
A Classical-Quantum Convolutional Neural Network for Detecting Pneumonia from Chest Radiographs V. Kulkarni S. Pawale A. Kharat 19 12 0 19 Feb 2022
Random Feature Amplification: Feature Learning and Generalization in Neural Networks Spencer Frei Niladri S. Chatterji Peter L. Bartlett MLT 30 29 0 15 Feb 2022
How does unlabeled data improve generalization in self-training? A one-hidden-layer theoretical analysis Shuai Zhang M. Wang Sijia Liu Pin-Yu Chen Jinjun Xiong SSL MLT 36 22 0 21 Jan 2022
Over-Parametrized Matrix Factorization in the Presence of Spurious Stationary Points Armin Eftekhari 19 1 0 25 Dec 2021
Recur, Attend or Convolve? On Whether Temporal Modeling Matters for Cross-Domain Robustness in Action Recognition Sofia Broomé Ernest Pokropek Boyu Li Hedvig Kjellström 13 7 0 22 Dec 2021
On the Convergence of Shallow Neural Network Training with Randomly Masked Neurons Fangshuo Liao Anastasios Kyrillidis 27 16 0 05 Dec 2021
Provable Regret Bounds for Deep Online Learning and Control Xinyi Chen Edgar Minasyan Jason D. Lee Elad Hazan 21 6 0 15 Oct 2021
Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networks Shuai Zhang Meng Wang Sijia Liu Pin-Yu Chen Jinjun Xiong UQCV MLT 13 13 0 12 Oct 2021
On the Global Convergence of Gradient Descent for multi-layer ResNets in the mean-field regime Zhiyan Ding Shi Chen Qin Li S. Wright MLT AI4CE 30 11 0 06 Oct 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 173 411 0 14 Jul 2021
Tensor Methods in Computer Vision and Deep Learning Yannis Panagakis Jean Kossaifi Grigorios G. Chrysos James Oldfield M. Nicolaou Anima Anandkumar S. Zafeiriou 19 119 0 07 Jul 2021
The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective Geoff Pleiss John P. Cunningham 23 24 0 11 Jun 2021
Understanding Overparameterization in Generative Adversarial Networks Yogesh Balaji M. Sajedi N. Kalibhat Mucong Ding Dominik Stöger Mahdi Soltanolkotabi S. Feizi AI4CE 12 21 0 12 Apr 2021
Landscape analysis for shallow neural networks: complete classification of critical points for affine target functions Patrick Cheridito Arnulf Jentzen Florian Rossmannek 16 10 0 19 Mar 2021
Recent Advances on Neural Network Pruning at Initialization Huan Wang Can Qin Yue Bai Yulun Zhang Yun Fu CVBM 31 64 0 11 Mar 2021
Fundamental Tradeoffs in Distributionally Adversarial Training M. Mehrabi Adel Javanmard Ryan A. Rossi Anup B. Rao Tung Mai AAML 20 17 0 15 Jan 2021
Align, then memorise: the dynamics of learning with feedback alignment Maria Refinetti Stéphane dÁscoli Ruben Ohana Sebastian Goldt 23 36 0 24 Nov 2020
Are wider nets better given the same number of parameters? A. Golubeva Behnam Neyshabur Guy Gur-Ari 11 44 0 27 Oct 2020
The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Training Andrea Montanari Yiqiao Zhong 20 95 0 25 Jul 2020
Quantitative Propagation of Chaos for SGD in Wide Neural Networks Valentin De Bortoli Alain Durmus Xavier Fontaine Umut Simsekli 16 25 0 13 Jul 2020
The Gaussian equivalence of generative models for learning with shallow neural networks Sebastian Goldt Bruno Loureiro Galen Reeves Florent Krzakala M. Mézard Lenka Zdeborová BDL 33 100 0 25 Jun 2020
Non-convergence of stochastic gradient descent in the training of deep neural networks Patrick Cheridito Arnulf Jentzen Florian Rossmannek 12 37 0 12 Jun 2020
Feature Purification: How Adversarial Training Performs Robust Deep Learning Zeyuan Allen-Zhu Yuanzhi Li MLT AAML 19 146 0 20 May 2020
Symmetry & critical points for a model shallow neural network Yossi Arjevani M. Field 26 13 0 23 Mar 2020
On Interpretability of Artificial Neural Networks: A Survey Fenglei Fan Jinjun Xiong Mengzhou Li Ge Wang AAML AI4CE 30 300 0 08 Jan 2020