Transport Monte Carlo

Markov chain Monte Carlo is routinely used for posterior estimation in Bayesian models; however, it can suffer from computing inefficiency, especially in high dimensional or hierarchical models, due to the high correlation appearing in the Markov chain. While approximate solutions have become popular, there are concerns about accuracy. Inspired by the optimal transport literature, we propose a new posterior estimation strategy by instead solving for a random transport plan between the target posterior and multivariate uniform distribution. Specifically, the uniform can be well approximated by an infinite mixture of one-to-one transforms from the posterior -- the reverse conditional is the posterior as a random draw from the transforms of the uniform, providing a way of rapidly generating independent posterior samples. Most importantly, via the Bayes' theorem, the drawing is directly weighted by the posterior density/mass function, leading to high approximation accuracy. Compared to the other inverse methods, our random transport plan is very simple to parameterize, such as a mixture of basic location-and-scale changes. We provide theoretic justifications and quantify the approximation error of the finite sample. Our method shows compelling advantages in the accuracy compared to other state-of-art approaches, and we demonstrate its practical usefulness in solving challenging problems, such estimating multi-modal distribution, high-dimensional sparse regression, and combinatorial graph.