A Complete Recipe for Stochastic Gradient MCMC

Many recent Markov chain Monte Carlo (MCMC) samplers leverage stochastic dynamics with state adaptation to define a Markov transition kernel that efficiently explores a target distribution. In tandem, a focus has been on devising scalable MCMC algorithms via data subsampling and using stochastic gradients in the stochastic dynamic simulations. However, such stochastic gradient MCMC methods have used simple stochastic dynamics, or required significant physical intuition to modify the dynamical system to account for the stochastic gradient noise. In this paper, we provide a general recipe for constructing MCMC samplers--including stochastic gradient versions--based on continuous Markov processes specified via two matrices. We constructively prove that the framework is complete. That is, any continuous Markov process that provides samples from the target distribution can be written in our framework. We demonstrate the utility of our recipe by trivially "reinventing" previously proposed stochastic gradient MCMC samplers, and in proposing a new state-adaptive sampler: stochastic gradient Riemann Hamiltonian Monte Carlo (SGRHMC). Our experiments on simulated data and a streaming Wikipedia analysis demonstrate that the proposed sampler inherits the benefits of Riemann HMC, with the scalability of stochastic gradient methods.