A Variational Analysis of Stochastic Gradient Algorithms

8 February 2016

Matthew D. Hoffman

Papers citing "A Variational Analysis of Stochastic Gradient Algorithms"

36 / 86 papers shown

Title
On the Heavy-Tailed Theory of Stochastic Gradient Descent for Deep Neural Networks Umut Simsekli Mert Gurbuzbalaban T. H. Nguyen G. Richard Levent Sagun 88 59 0 29 Nov 2019
Thompson Sampling via Local Uncertainty Zhendong Wang Mingyuan Zhou 80 19 0 30 Oct 2019
Non-Gaussianity of Stochastic Gradient Noise A. Panigrahi Raghav Somani Navin Goyal Praneeth Netrapalli 68 53 0 21 Oct 2019
Stochastic Gradient and Langevin Processes Xiang Cheng Dong Yin Peter L. Bartlett Michael I. Jordan 82 5 0 07 Jul 2019
First Exit Time Analysis of Stochastic Gradient Descent Under Heavy-Tailed Gradient Noise T. H. Nguyen Umut Simsekli Mert Gurbuzbalaban G. Richard 79 65 0 21 Jun 2019
Adaptive Temporal-Difference Learning for Policy Evaluation with Per-State Uncertainty Estimates Hugo Penedones C. Riquelme Damien Vincent Hartmut Maennel Timothy A. Mann André Barreto Sylvain Gelly Gergely Neu OffRL 46 10 0 19 Jun 2019
Multi-Class Gaussian Process Classification Made Conjugate: Efficient Inference via Data Augmentation Théo Galy-Fajou F. Wenzel Christian Donner Manfred Opper 60 30 0 23 May 2019
Distribution-Dependent Analysis of Gibbs-ERM Principle Ilja Kuzborskij Nicolò Cesa-Bianchi Csaba Szepesvári 85 20 0 05 Feb 2019
Quantitative Weak Convergence for Discrete Stochastic Processes Xiang Cheng Peter L. Bartlett Michael I. Jordan 43 5 0 03 Feb 2019
Quasi-potential as an implicit regularizer for the loss function in the stochastic gradient descent Wenqing Hu Zhanxing Zhu Haoyi Xiong Jun Huan MLT 54 10 0 18 Jan 2019
A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks Umut Simsekli Levent Sagun Mert Gurbuzbalaban 114 252 0 18 Jan 2019
A continuous-time analysis of distributed stochastic gradient Nicholas M. Boffi Jean-Jacques E. Slotine 46 15 0 28 Dec 2018
Information-Directed Exploration for Deep Reinforcement Learning Nikolay Nikolov Johannes Kirschner Felix Berkenkamp Andreas Krause 67 72 0 18 Dec 2018
Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations Qianxiao Li Cheng Tai E. Weinan 124 150 0 05 Nov 2018
Continuous-time Models for Stochastic Optimization Algorithms Antonio Orvieto Aurelien Lucchi 119 32 0 05 Oct 2018
Diffusion Approximations for Online Principal Component Estimation and Global Convergence C. J. Li Mengdi Wang Han Liu Tong Zhang 82 12 0 29 Aug 2018
Online ICA: Understanding Global Dynamics of Nonconvex Optimization via Diffusion Processes C. J. Li Zhaoran Wang Han Liu DiffM 113 18 0 29 Aug 2018
PCA of high dimensional random walks with comparison to neural network training J. Antognini Jascha Narain Sohl-Dickstein OOD 62 29 0 22 Jun 2018
Differential Equations for Modeling Asynchronous Algorithms Li He Qi Meng Wei-neng Chen Zhiming Ma Tie-Yan Liu 52 9 0 08 May 2018
Scalable Generalized Dynamic Topic Models P. Jähnichen F. Wenzel Marius Kloft Stephan Mandt BDL 107 40 0 21 Mar 2018
Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling C. Riquelme George Tucker Jasper Snoek BDL 93 366 0 26 Feb 2018
Efficient Gaussian Process Classification Using Polya-Gamma Data Augmentation F. Wenzel Théo Galy-Fajou Christian Donner Marius Kloft Manfred Opper 105 36 0 18 Feb 2018
Asymptotic Analysis via Stochastic Differential Equations of Gradient Descent Algorithms in Statistical and Computational Paradigms Yazhen Wang 93 17 0 27 Nov 2017
Advances in Variational Inference Cheng Zhang Judith Butepage Hedvig Kjellström Stephan Mandt BDL 233 698 0 15 Nov 2017
Stochastic gradient descent performs variational inference, converges to limit cycles for deep networks Pratik Chaudhari Stefano Soatto MLT 104 304 0 30 Oct 2017
Learning to Draw Samples with Amortized Stein Variational Gradient Descent Yihao Feng Dilin Wang Qiang Liu GAN BDL 85 82 0 20 Jul 2017
Bridging the Gap between Constant Step Size Stochastic Gradient Descent and Markov Chains Aymeric Dieuleveut Alain Durmus Francis R. Bach 108 156 0 20 Jul 2017
Bayesian Nonlinear Support Vector Machines for Big Data F. Wenzel Théo Galy-Fajou M. Deutsch Marius Kloft BDL 73 27 0 18 Jul 2017
A Divergence Bound for Hybrids of MCMC and Variational Inference and an Application to Langevin Dynamics and SGVI Justin Domke BDL 80 6 0 20 Jun 2017
Statistical inference using SGD Tianyang Li Liu Liu Anastasios Kyrillidis Constantine Caramanis FedML 58 38 0 21 May 2017
Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples Haw-Shiuan Chang Erik Learned-Miller Andrew McCallum 100 355 0 24 Apr 2017
Approximate Inference with Amortised MCMC Yingzhen Li Richard Turner Qiang Liu BDL 96 62 0 27 Feb 2017
Two Methods For Wild Variational Inference Qiang Liu Yihao Feng BDL 138 19 0 30 Nov 2016
Entropy-SGD: Biasing Gradient Descent Into Wide Valleys Pratik Chaudhari A. Choromańska Stefano Soatto Yann LeCun Carlo Baldassi C. Borgs J. Chayes Levent Sagun R. Zecchina ODL 118 775 0 06 Nov 2016
Patterns of Scalable Bayesian Inference E. Angelino Matthew J. Johnson Ryan P. Adams 107 87 0 16 Feb 2016
Stochastic modified equations and adaptive stochastic gradient algorithms Qianxiao Li Cheng Tai E. Weinan 77 285 0 19 Nov 2015