Small nonlinearities in activation functions create bad local minima in
neural networks

Small nonlinearities in activation functions create bad local minima in neural networks

10 February 2018

Papers citing "Small nonlinearities in activation functions create bad local minima in neural networks"

14 / 14 papers shown

Title
Read the Signs: Towards Invariance to Gradient Descent's Hyperparameter Initialization Davood Wadi M. Fredette S. Sénécal ODL AI4CE 6 0 0 24 Jan 2023
When Expressivity Meets Trainability: Fewer than $n$ Neurons Can Work Jiawei Zhang Yushun Zhang Mingyi Hong Ruoyu Sun Z. Luo 26 10 0 21 Oct 2022
Exact Solutions of a Deep Linear Network Liu Ziyin Botao Li Xiangmin Meng ODL 19 21 0 10 Feb 2022
Exponentially Many Local Minima in Quantum Neural Networks Xuchen You Xiaodi Wu 69 51 0 06 Oct 2021
Review: Deep Learning in Electron Microscopy Jeffrey M. Ede 29 79 0 17 Sep 2020
A Mean-field Analysis of Deep ResNet and Beyond: Towards Provable Optimization Via Overparameterization From Depth Yiping Lu Chao Ma Yulong Lu Jianfeng Lu Lexing Ying MLT 33 78 0 11 Mar 2020
The Usual Suspects? Reassessing Blame for VAE Posterior Collapse Bin Dai Ziyu Wang David Wipf DRL 14 75 0 23 Dec 2019
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruosong Wang MLT 35 961 0 24 Jan 2019
Width Provably Matters in Optimization for Deep Linear Neural Networks S. Du Wei Hu 16 93 0 24 Jan 2019
Non-attracting Regions of Local Minima in Deep and Wide Neural Networks Henning Petzka C. Sminchisescu 25 9 0 16 Dec 2018
Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity Chulhee Yun S. Sra Ali Jadbabaie 13 117 0 17 Oct 2018
Learning ReLU Networks on Linearly Separable Data: Algorithm, Optimality, and Generalization G. Wang G. Giannakis Jie Chen MLT 22 131 0 14 Aug 2018
Global optimality conditions for deep neural networks Chulhee Yun S. Sra Ali Jadbabaie 125 117 0 08 Jul 2017
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 179 1,185 0 30 Nov 2014