Sharp Minima Can Generalize For Deep Nets

15 March 2017

Papers citing "Sharp Minima Can Generalize For Deep Nets"

32 / 132 papers shown

Title
GradVis: Visualization and Second Order Analysis of Optimization Surfaces during the Training of Deep Neural Networks Avraam Chatzimichailidis Franz-Josef Pfreundt N. Gauger J. Keuper 11 10 0 26 Sep 2019
Empirical study of extreme overfitting points of neural networks D. Merkulov Ivan V. Oseledets 3DPC 6 7 0 14 Jun 2019
Orthogonal Deep Neural Networks K. Jia Shuai Li Yuxin Wen Tongliang Liu Dacheng Tao 26 131 0 15 May 2019
Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization Hesham Mostafa Xin Wang 18 307 0 15 Feb 2019
Ensemble Feature for Person Re-Identification Jiabao Wang Yang Li Zhuang Miao OOD 3DPC 8 1 0 17 Jan 2019
Towards Theoretical Understanding of Large Batch Training in Stochastic Gradient Descent Xiaowu Dai Yuhua Zhu 17 11 0 03 Dec 2018
Sequenced-Replacement Sampling for Deep Learning C. Ho Dae Hoon Park Wei Yang Yi Chang 19 0 0 19 Oct 2018
Generalization and Regularization in DQN Jesse Farebrother Marlos C. Machado Michael H. Bowling 23 203 0 29 Sep 2018
Don't Use Large Mini-Batches, Use Local SGD Tao R. Lin Sebastian U. Stich Kumar Kshitij Patel Martin Jaggi 16 429 0 22 Aug 2018
Understanding training and generalization in deep learning by Fourier analysis Zhi-Qin John Xu AI4CE 11 92 0 13 Aug 2018
Generalization Error in Deep Learning Daniel Jakubovitz Raja Giryes M. Rodrigues AI4CE 16 109 0 03 Aug 2018
PCA of high dimensional random walks with comparison to neural network training J. Antognini Jascha Narain Sohl-Dickstein OOD 11 27 0 22 Jun 2018
On Tighter Generalization Bound for Deep Neural Networks: CNNs, ResNets, and Beyond Xingguo Li Junwei Lu Zhaoran Wang Jarvis D. Haupt T. Zhao 23 78 0 13 Jun 2018
SmoothOut: Smoothing Out Sharp Minima to Improve Generalization in Deep Learning W. Wen Yandan Wang Feng Yan Cong Xu Chunpeng Wu Yiran Chen H. Li 16 49 0 21 May 2018
The Loss Surface of XOR Artificial Neural Networks D. Mehta Xiaojun Zhao Edgar A. Bernal D. Wales 19 19 0 06 Apr 2018
Analysis on the Nonlinear Dynamics of Deep Neural Networks: Topological Entropy and Chaos Husheng Li 9 11 0 03 Apr 2018
On the importance of single directions for generalization Ari S. Morcos David Barrett Neil C. Rabinowitz M. Botvinick 11 328 0 19 Mar 2018
Averaging Weights Leads to Wider Optima and Better Generalization Pavel Izmailov Dmitrii Podoprikhin T. Garipov Dmitry Vetrov A. Wilson FedML MoMe 9 1,616 0 14 Mar 2018
Essentially No Barriers in Neural Network Energy Landscape Felix Dräxler K. Veschgini M. Salmhofer Fred Hamprecht MoMe 20 423 0 02 Mar 2018
A Walk with SGD Chen Xing Devansh Arpit Christos Tsirigotis Yoshua Bengio 17 118 0 24 Feb 2018
$$\mathcal{G}$-SGD: Optimizing ReLU Neural Networks in its Positively Scale-Invariant Space$ $\mathcal{G}$ -SGD: Optimizing ReLU Neural Networks in its Positively Scale-Invariant Space Qi Meng Shuxin Zheng Huishuai Zhang Wei-Neng Chen Zhi-Ming Ma Tie-Yan Liu 27 38 0 11 Feb 2018
Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers Jianbo Ye Xin Lu Zhe-nan Lin J. Z. Wang 11 405 0 01 Feb 2018
Theory of Deep Learning IIb: Optimization Properties of SGD Chiyuan Zhang Q. Liao Alexander Rakhlin Brando Miranda Noah Golowich T. Poggio ODL 15 70 0 07 Jan 2018
Visualizing the Loss Landscape of Neural Nets Hao Li Zheng Xu Gavin Taylor Christoph Studer Tom Goldstein 31 1,840 0 28 Dec 2017
Scale out for large minibatch SGD: Residual network training on ImageNet-1K with improved accuracy and reduced time to train V. Codreanu Damian Podareanu V. Saletore 23 54 0 12 Nov 2017
Rethinking generalization requires revisiting old ideas: statistical mechanics approaches and complex learning behavior Charles H. Martin Michael W. Mahoney AI4CE 19 62 0 26 Oct 2017
Train longer, generalize better: closing the generalization gap in large batch training of neural networks Elad Hoffer Itay Hubara Daniel Soudry ODL 25 792 0 24 May 2017
Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data Gintare Karolina Dziugaite Daniel M. Roy 31 797 0 31 Mar 2017
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Z. Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 716 6,740 0 26 Sep 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 273 2,886 0 15 Sep 2016
Understanding symmetries in deep networks Vijay Badrinarayanan Bamdev Mishra R. Cipolla 219 42 0 03 Nov 2015
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 175 1,185 0 30 Nov 2014