Train faster, generalize better: Stability of stochastic gradient descent

3 September 2015

Benjamin Recht

Papers citing "Train faster, generalize better: Stability of stochastic gradient descent"

49 / 199 papers shown

Title
Stochastic Iterative Hard Thresholding for Graph-structured Sparsity Optimization Baojian Zhou F. Chen Yiming Ying 21 7 0 09 May 2019
Stability and Generalization of Graph Convolutional Neural Networks Saurabh Verma Zhi-Li Zhang GNN MLT 24 153 0 03 May 2019
Deep Multi-View Learning using Neuron-Wise Correlation-Maximizing Regularizers K. Jia Jiehong Lin Mingkui Tan Dacheng Tao 3DV 19 32 0 25 Apr 2019
A Selective Overview of Deep Learning Jianqing Fan Cong Ma Yiqiao Zhong BDL VLM 28 136 0 10 Apr 2019
High probability generalization bounds for uniformly stable algorithms with nearly optimal rate Vitaly Feldman J. Vondrák 16 154 0 27 Feb 2019
An Empirical Study of Large-Batch Stochastic Gradient Descent with Structured Covariance Noise Yeming Wen Kevin Luk Maxime Gazeau Guodong Zhang Harris Chan Jimmy Ba ODL 20 22 0 21 Feb 2019
Stable and Fair Classification Lingxiao Huang Nisheeth K. Vishnoi FaML 19 71 0 21 Feb 2019
Quasi-Newton Methods for Machine Learning: Forget the Past, Just Sample A. Berahas Majid Jahani Peter Richtárik Martin Takávc 11 40 0 28 Jan 2019
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruosong Wang MLT 35 961 0 24 Jan 2019
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du J. Lee Haochuan Li Liwei Wang M. Tomizuka ODL 21 1,120 0 09 Nov 2018
Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel Colin Wei J. Lee Qiang Liu Tengyu Ma 18 243 0 12 Oct 2018
Graph-Dependent Implicit Regularisation for Distributed Stochastic Subgradient Descent Dominic Richards Patrick Rebeschini 16 18 0 18 Sep 2018
On the Generalization of Stochastic Gradient Descent with Momentum Ali Ramezani-Kebrya Kimon Antonakopoulos V. Cevher Ashish Khisti Ben Liang MLT 12 23 0 12 Sep 2018
Understanding training and generalization in deep learning by Fourier analysis Zhi-Qin John Xu AI4CE 19 92 0 13 Aug 2018
Generalization Error in Deep Learning Daniel Jakubovitz Raja Giryes M. Rodrigues AI4CE 21 109 0 03 Aug 2018
Laplacian Smoothing Gradient Descent Stanley Osher Bao Wang Penghang Yin Xiyang Luo Farzin Barekat Minh Pham A. Lin ODL 19 43 0 17 Jun 2018
PAC-Bayes Control: Learning Policies that Provably Generalize to Novel Environments Anirudha Majumdar M. Goldstein Anoopkumar Sonar 20 18 0 11 Jun 2018
Training Faster by Separating Modes of Variation in Batch-normalized Models Mahdi M. Kalayeh M. Shah 19 42 0 07 Jun 2018
Measuring and regularizing networks in function space Ari S. Benjamin David Rolnick Konrad Paul Kording 21 137 0 21 May 2018
Stochastic modified equations for the asynchronous stochastic gradient descent Jing An Jian-wei Lu Lexing Ying 16 79 0 21 May 2018
Constrained-CNN losses for weakly supervised segmentation H. Kervadec Jose Dolz Meng Tang Eric Granger Yuri Boykov Ismail Ben Ayed 27 239 0 12 May 2018
Privacy-preserving Prediction Cynthia Dwork Vitaly Feldman 6 90 0 27 Mar 2018
Constrained Deep Learning using Conditional Gradient and Applications in Computer Vision Sathya Ravi Tuan Dinh Vishnu Suresh Lokhande Vikas Singh AI4CE 18 22 0 17 Mar 2018
A Walk with SGD Chen Xing Devansh Arpit Christos Tsirigotis Yoshua Bengio 17 118 0 24 Feb 2018
Learning Compact Neural Networks with Regularization Samet Oymak MLT 35 39 0 05 Feb 2018
Generalization Error Bounds for Noisy, Iterative Algorithms Ankit Pensia Varun Jog Po-Ling Loh 10 109 0 12 Jan 2018
Convergence of Unregularized Online Learning Algorithms Yunwen Lei Lei Shi Zheng-Chu Guo 14 14 0 09 Aug 2017
Regularizing and Optimizing LSTM Language Models Stephen Merity N. Keskar R. Socher 54 1,090 0 07 Aug 2017
Stochastic Training of Neural Networks via Successive Convex Approximations Simone Scardapane P. Di Lorenzo 14 9 0 15 Jun 2017
Recovery Guarantees for One-hidden-layer Neural Networks Kai Zhong Zhao-quan Song Prateek Jain Peter L. Bartlett Inderjit S. Dhillon MLT 17 335 0 10 Jun 2017
Are Saddles Good Enough for Deep Learning? Adepu Ravi Sankar V. Balasubramanian 30 5 0 07 Jun 2017
Train longer, generalize better: closing the generalization gap in large batch training of neural networks Elad Hoffer Itay Hubara Daniel Soudry ODL 27 792 0 24 May 2017
The Marginal Value of Adaptive Gradient Methods in Machine Learning Ashia C. Wilson Rebecca Roelofs Mitchell Stern Nathan Srebro Benjamin Recht ODL 6 1,012 0 23 May 2017
Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data Gintare Karolina Dziugaite Daniel M. Roy 48 799 0 31 Mar 2017
Sharp Minima Can Generalize For Deep Nets Laurent Dinh Razvan Pascanu Samy Bengio Yoshua Bengio ODL 37 754 0 15 Mar 2017
Fast Rates for Empirical Risk Minimization of Strict Saddle Problems Alon Gonen Shai Shalev-Shwartz 33 29 0 16 Jan 2017
Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond Levent Sagun Léon Bottou Yann LeCun UQCV 16 227 0 22 Nov 2016
Learning Scalable Deep Kernels with Recurrent Structure Maruan Al-Shedivat A. Wilson Yunus Saatchi Zhiting Hu Eric P. Xing BDL 13 104 0 27 Oct 2016
Membership Inference Attacks against Machine Learning Models Reza Shokri M. Stronati Congzheng Song Vitaly Shmatikov SLR MIALM MIACV 30 4,021 0 18 Oct 2016
AdaNet: Adaptive Structural Learning of Artificial Neural Networks Corinna Cortes X. Gonzalvo Vitaly Kuznetsov M. Mohri Scott Yang 21 282 0 05 Jul 2016
On the Expressive Power of Deep Neural Networks M. Raghu Ben Poole Jon M. Kleinberg Surya Ganguli Jascha Narain Sohl-Dickstein 25 777 0 16 Jun 2016
View-tolerant face recognition and Hebbian learning imply mirror-symmetric neural tuning to head orientation Joel Z. Leibo Q. Liao W. Freiwald Fabio Anselmi T. Poggio CVBM 16 56 0 05 Jun 2016
Deep Q-Networks for Accelerating the Training of Deep Neural Networks Jie Fu AI4CE 23 11 0 05 Jun 2016
Fast Zero-Shot Image Tagging Yang Zhang Boqing Gong M. Shah VLM 3DV 14 141 0 31 May 2016
Alternative asymptotics for cointegration tests in large VARs Junhong Lin Lorenzo Rosasco 15 43 0 28 May 2016
No bad local minima: Data independent training error guarantees for multilayer neural networks Daniel Soudry Y. Carmon 17 235 0 26 May 2016
Swapout: Learning an ensemble of deep architectures Saurabh Singh Derek Hoiem David A. Forsyth BDL 3DPC OOD UQCV 17 150 0 20 May 2016
Stabilized Sparse Online Learning for Sparse Data Yuting Ma Tian Zheng 15 14 0 21 Apr 2016
Ensemble Robustness and Generalization of Stochastic Deep Learning Algorithms Tom Zahavy Bingyi Kang Alex Sivak Jiashi Feng Huan Xu Shie Mannor OOD AAML 31 12 0 07 Feb 2016