Adding One Neuron Can Eliminate All Bad Local Minima

22 May 2018

Papers citing "Adding One Neuron Can Eliminate All Bad Local Minima"

50 / 50 papers shown

Title
Flat Channels to Infinity in Neural Loss Landscapes Flavio Martinelli Alexander Van Meegen Berfin Simsek W. Gerstner Johanni Brea 269 2 0 17 Jun 2025
Analysis of the rate of convergence of an over-parametrized convolutional neural network image classifier learned by gradient descentJournal of Statistical Planning and Inference (JSPI), 2024 Michael Kohler A. Krzyżak Benjamin Walter 219 1 0 13 May 2024
Exploring Neural Network Landscapes: Star-Shaped and Geodesic Connectivity Zhanran Lin Puheng Li Lei Wu 423 9 0 09 Apr 2024
Blurry Video Compression: A Trade-off between Visual Enhancement and Data Compression Dawit Mureja Argaw Junsik Kim In So Kweon 126 1 0 08 Nov 2023
Global Optimality in Bivariate Gradient-based DAG LearningNeural Information Processing Systems (NeurIPS), 2023 Chang Deng Kevin Bello Bryon Aragam Pradeep Ravikumar 182 11 0 30 Jun 2023
NTK-SAP: Improving neural network pruning by aligning training dynamicsInternational Conference on Learning Representations (ICLR), 2023 Yite Wang Dawei Li Tian Ding 195 31 0 06 Apr 2023
When Expressivity Meets Trainability: Fewer than $n$ Neurons Can WorkNeural Information Processing Systems (NeurIPS), 2022 Jiawei Zhang Yushun Zhang Mingyi Hong Tian Ding Jianfeng Yao 279 10 0 21 Oct 2022
Noise Regularizes Over-parameterized Rank One Matrix Recovery, ProvablyInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2022 Tianyi Liu Yan Li Enlu Zhou Tuo Zhao 119 1 0 07 Feb 2022
Global Convergence Analysis of Deep Linear Networks with A One-neuron Layer Kun Chen Dachao Lin Zhihua Zhang 136 1 0 08 Jan 2022
Theoretical Exploration of Flexible Transmitter ModelIEEE Transactions on Neural Networks and Learning Systems (TNNLS), 2021 Jin-Hui Wu Shao-Qun Zhang Yuan Jiang Zhiping Zhou 339 3 0 11 Nov 2021
Convergence rates for shallow neural networks learned by gradient descent Alina Braun Michael Kohler S. Langer Harro Walk 175 14 0 20 Jul 2021
A Geometric Analysis of Neural Collapse with Unconstrained FeaturesNeural Information Processing Systems (NeurIPS), 2021 Zhihui Zhu Tianyu Ding Jinxin Zhou Xiao Li Chong You Jeremias Sulam Qing Qu 266 230 0 06 May 2021
Achieving Small Test Error in Mildly Overparameterized Neural Networks Shiyu Liang Tian Ding R. Srikant 169 3 0 24 Apr 2021
Training Deep Neural Networks via Branch-and-Bound Yuanwei Wu Ziming Zhang Guanghui Wang ODL 230 0 0 05 Apr 2021
Spurious Local Minima Are Common for Deep Neural Networks with Piecewise Linear ActivationsIEEE Transactions on Neural Networks and Learning Systems (TNNLS), 2021 Bo Liu 123 9 0 25 Feb 2021
When Are Solutions Connected in Deep Networks?Neural Information Processing Systems (NeurIPS), 2021 Quynh N. Nguyen Pierre Bréchet Marco Mondelli 349 10 0 18 Feb 2021
WGAN with an Infinitely Wide Generator Has No Spurious Stationary PointsInternational Conference on Machine Learning (ICML), 2021 Albert No Taeho Yoon Sehyun Kwon Ernest K. Ryu GAN 163 2 0 15 Feb 2021
On the Theory of Implicit Deep Learning: Global Convergence with Implicit LayersInternational Conference on Learning Representations (ICLR), 2021 Kenji Kawaguchi PINN 194 44 0 15 Feb 2021
A Comprehensive Study on Optimization Strategies for Gradient Descent In Deep Learning K. Yadav 108 1 0 07 Jan 2021
Towards a Better Global Loss Landscape of GANs Tian Ding Tiantian Fang Alex Schwing GAN 203 32 0 10 Nov 2020
From Symmetry to Geometry: Tractable Nonconvex Problems Yuqian Zhang Qing Qu John N. Wright 287 48 0 14 Jul 2020
Maximum-and-Concatenation NetworksInternational Conference on Machine Learning (ICML), 2020 Xingyu Xie Hao Kong Yue Yu Wayne Zhang Guangcan Liu Zhouchen Lin 244 2 0 09 Jul 2020
The Global Landscape of Neural Networks: An Overview Tian Ding Dawei Li Shiyu Liang Tian Ding R. Srikant 186 92 0 02 Jul 2020
Global Convergence and Generalization Bound of Gradient-Based Meta-Learning with Deep Neural Nets Haoxiang Wang Tian Ding Bo Li MLT AI4CE 201 14 0 25 Jun 2020
Escaping Saddle Points Efficiently with Occupation-Time-Adapted Perturbations Xin Guo Jiequn Han Mahan Tajrobehkar Wenpin Tang 189 3 0 09 May 2020
Some Geometrical and Topological Properties of DNNs' Decision Boundaries Bo Liu Mengya Shen AAML 188 3 0 07 Mar 2020
Understanding Global Loss Landscape of One-hidden-layer ReLU Networks, Part 1: Theory Bo Liu FAtt MLT 189 1 0 12 Feb 2020
Revisiting Landscape Analysis in Deep Neural Networks: Eliminating Decreasing Paths to InfinitySIAM Journal on Optimization (SIOPT), 2019 Shiyu Liang Tian Ding R. Srikant 140 21 0 31 Dec 2019
Landscape Connectivity and Dropout Stability of SGD Solutions for Over-parameterized Neural NetworksInternational Conference on Machine Learning (ICML), 2019 Aleksandr Shevchenko Marco Mondelli 358 41 0 20 Dec 2019
Optimization for deep learning: theory and algorithms Tian Ding ODL 275 177 0 19 Dec 2019
Over-parametrized deep neural networks do not generalize well Michael Kohler A. Krzyżak 133 15 0 09 Dec 2019
Analysis of the rate of convergence of neural network regression estimates which are easy to implement Alina Braun Michael Kohler A. Krzyżak 139 1 0 09 Dec 2019
On the rate of convergence of a neural network regression estimate learned by gradient descent Alina Braun Michael Kohler Harro Walk 99 11 0 09 Dec 2019
Sub-Optimal Local Minima Exist for Neural Networks with Almost All Non-Linear Activations Tian Ding Dawei Li Tian Ding 285 14 0 04 Nov 2019
The Local Elasticity of Neural NetworksInternational Conference on Learning Representations (ICLR), 2019 Hangfeng He Weijie J. Su 262 51 0 15 Oct 2019
Dynamics of Deep Neural Networks and Neural Tangent HierarchyInternational Conference on Machine Learning (ICML), 2019 Jiaoyang Huang H. Yau 143 160 0 18 Sep 2019
Are deep ResNets provably better than linear predictors?Neural Information Processing Systems (NeurIPS), 2019 Chulhee Yun S. Sra Ali Jadbabaie 243 13 0 09 Jul 2019
Training CNNs with Selective Allocation of ChannelsInternational Conference on Machine Learning (ICML), 2019 Jongheon Jeong Jinwoo Shin CVBM 176 16 0 11 May 2019
Traversing the noise of dynamic mini-batch sub-sampled loss functions: A visual guide D. Kafka D. Wilke 106 0 0 20 Mar 2019
Mean Field Analysis of Deep Neural Networks Justin A. Sirignano K. Spiliopoulos 204 85 0 11 Mar 2019
Uniform-in-Time Weak Error Analysis for Stochastic Gradient Descent Algorithms via Diffusion ApproximationCommunications in Mathematical Sciences (Comm. Math. Sci.), 2019 Yuanyuan Feng Tingran Gao Lei Li Jian‐Guo Liu Yulong Lu 167 25 0 02 Feb 2019
On Connected Sublevel Sets in Deep Learning Quynh N. Nguyen 263 106 0 22 Jan 2019
Eliminating all bad Local Minima from Loss Landscapes without even adding an Extra Unit Jascha Narain Sohl-Dickstein Kenji Kawaguchi 124 6 0 12 Jan 2019
Elimination of All Bad Local Minima in Deep Learning Kenji Kawaguchi L. Kaelbling 272 47 0 02 Jan 2019
Non-attracting Regions of Local Minima in Deep and Wide Neural Networks Henning Petzka C. Sminchisescu 209 12 0 16 Dec 2018
On the loss landscape of a class of deep neural networks with no bad local valleysInternational Conference on Learning Representations (ICLR), 2018 Quynh N. Nguyen Mahesh Chandra Mukkamala Matthias Hein 331 89 0 27 Sep 2018
Learning ReLU Networks on Linearly Separable Data: Algorithm, Optimality, and Generalization G. Wang G. Giannakis Jie Chen MLT 170 144 0 14 Aug 2018
Deep Neural Networks with Multi-Branch Architectures Are Less Non-Convex Hongyang R. Zhang Junru Shao Ruslan Salakhutdinov 232 14 0 06 Jun 2018
Small nonlinearities in activation functions create bad local minima in neural networks Chulhee Yun S. Sra Ali Jadbabaie ODL 289 97 0 10 Feb 2018
McKernel: A Library for Approximate Kernel Expansions in Log-linear Time J. Curtò I. Zarza Feng Yang Alex Smola Fernando de la Torre Chong Wah Ngo Luc van Gool 510 3 0 27 Feb 2017