Transport Monte Carlo

In Bayesian posterior estimation, the transport map finds a deterministic transform from a simple reference distribution to a potentially complicated posterior distribution. Compared to other sampling approaches, it is capable of generating independent samples while exploiting efficient optimization toolboxes. However, a fundamental concern is that the invertible map is challenging to parameterize with sufficient flexibility, and may even fail to exist between the two distributions. To address this issue, we propose Transport Monte Carlo that parameterizes the transform as a random choice from several maps. It corresponds to a coupling distribution of the reference and posterior, which is guaranteed to exist under mild conditions. This framework allows us to decompose a sophisticated transform into multiple maps; each is now simple to parameterize and estimate. In the meantime, it allows a direct extension to coupling a continuous reference and a discrete posterior. We examine its theoretical properties, including the error rate due to the finite training sample size. Compared to existing methods such as Hamiltonian Monte Carlo or neural network-based transport map, our method demonstrates much-improved performances in several common sampling problems, including the multi-modal distribution, high-dimensional sparse regression, and combinatorial sampling of graph edges.