Ergodicity of Approximate MCMC Chains with Applications to Large Data Sets

In many modern applications, difficulty in evaluating the posterior density makes taking performing a single MCMC step slow; this difficulty can be caused by intractable likelihood functions, but also appears for routine problems with large data sets. Many researchers have responded by running approximate versions of MCMC algorithms. In this note, we develop very general quantitative bounds for showing the ergodicity of these approximate samplers. In particular, our bounds can be used to perform a bias-variance trade-off argument to give practical guidance on the quality of approximation that should be used for a given total computational budget. We present a few applications of our results in recently proposed algorithms, including the "austerity" framework, Stochastic Gradient Langevin Dynamics, exponential random graphs and ABC-MCMC algorithms.