Markov Chain Monte Carlo and Variational Inference: Bridging the Gap

Recent advances in stochastic variational inference have made it possible to construct variational posterior approximations containing auxiliary random variables. This enables us to explore a new synthesis of variational inference and Monte Carlo methods where we incorporate one or more steps of MCMC into our variational approximation. This note describes the theoretical foundations that make this possible and shows some promising first results.