Stochastic gradient descent performs variational inference, converges to
limit cycles for deep networks

v1v2 (latest)

Stochastic gradient descent performs variational inference, converges to limit cycles for deep networks

30 October 2017

Pratik Chaudhari

ArXiv (abs)PDF HTML

Papers citing "Stochastic gradient descent performs variational inference, converges to limit cycles for deep networks"

12 / 112 papers shown

Title
On the Spectral Bias of Neural Networks Nasim Rahaman A. Baratin Devansh Arpit Felix Dräxler Min Lin Fred Hamprecht Yoshua Bengio Aaron Courville 170 1,460 0 22 Jun 2018
Stochastic Gradient/Mirror Descent: Minimax Optimality and Implicit Regularization Navid Azizan B. Hassibi 86 64 0 04 Jun 2018
Understanding Batch Normalization Johan Bjorck Carla P. Gomes B. Selman Kilian Q. Weinberger 177 617 0 01 Jun 2018
SaaS: Speed as a Supervisor for Semi-supervised Learning Safa Cicek Alhussein Fawzi Stefano Soatto BDL 85 19 0 02 May 2018
Escaping Saddles with Stochastic Gradients Hadi Daneshmand Jonas Köhler Aurelien Lucchi Thomas Hofmann 75 162 0 15 Mar 2018
A Walk with SGD Chen Xing Devansh Arpit Christos Tsirigotis Yoshua Bengio 96 119 0 24 Feb 2018
Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problem Andre Wibisono 123 183 0 22 Feb 2018
An Alternative View: When Does SGD Escape Local Minima? Robert D. Kleinberg Yuanzhi Li Yang Yuan MLT 93 317 0 17 Feb 2018
Three Factors Influencing Minima in SGD Stanislaw Jastrzebski Zachary Kenton Devansh Arpit Nicolas Ballas Asja Fischer Yoshua Bengio Amos Storkey 85 463 0 13 Nov 2017
Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates L. Smith Nicholay Topin AI4CE 106 520 0 23 Aug 2017
Emergence of Invariance and Disentanglement in Deep Representations Alessandro Achille Stefano Soatto OOD DRL 122 479 0 05 Jun 2017
Deep Relaxation: partial differential equations for optimizing deep neural networks Pratik Chaudhari Adam M. Oberman Stanley Osher Stefano Soatto G. Carlier 158 154 0 17 Apr 2017