Non-Reversible Parallel Tempering: a Scalable Highly Parallel MCMC Scheme

Parallel tempering (PT) methods are a popular class of Markov chain Monte Carlo schemes used to sample complex high-dimensional probability distributions. They rely on a collection of $N$ interacting auxiliary chains targeting tempered versions of the target distribution to improve the exploration of the state-space. We provide here a new perspective on these highly parallel algorithms and their tuning by identifying and formalizing a sharp divide in the behaviour and performance of reversible versus non-reversible PT schemes. By analyzing the behaviour of PT algorithms using a novel asymptotic regime in which $N$ goes to infinity, we show indeed that a class of non-reversible PT methods dominates its reversible counterparts and identify distinct scaling limits for the non-reversible and reversible schemes, the former being a piecewise-deterministic Markov process and the latter a diffusion. In particular, a major limitation of reversible PT is that its performances eventually collapse as $N$ increases whereas those of non-reversible PT improve. These theoretical results are exploited to develop an adaptive non-reversible PT scheme approximating the optimal annealing schedule. We provide a wide range of numerical examples supporting our theoretical and methodological contributions.