Beyond the Central Limit Theorem: Universal and Non-universal Simulations of Random Variables by General Mappings

The Central Limit Theorem states that a standard Gaussian random variable can be simulated within any level of approximation error (measured by the Kolmogorov-Smirnov distance) from an i.i.d. real-valued random vector $X^{n}\sim P_{X}^{n}$ by a normalized sum mapping (as $n\to\infty$). Moreover given the mean and variance of $X$, this linear function is independent of the distribution $P_{X}$. Such simulation problems (in which the simulation mappings are independent of $P_{X}$, or equivalently $P_{X}$ is unknown a prior) are referred to as being universal. In this paper, we consider both universal and non-universal simulations of random variables with arbitrary target distributions $Q_{Y}$ by general mappings, not limited to linear ones. We derive the fastest convergence rate of the approximation errors for such problems. Interestingly, we show that for discontinuous or absolutely continuous $P_{X}$, the approximation error for the universal simulation is almost as small as that for the non-universal one; and moreover, for both universal and non-universal simulations, the approximation errors by general mappings are strictly smaller than those by linear mappings. Specifically, for both universal and non-universal simulations, if $P_{X}$ is discontinuous, then the approximation error decays at least exponentially fast as $n\to\infty$; if $P_{X}$ is absolutely continuous, then only one-dimensional $X$ is sufficient to simulate $Y$ exactly or arbitrarily well. For continuous but not absolutely continuous $P_{X}$, using a non-universal simulator, one-dimensional $X$ is still sufficient to simulate $Y$ exactly, however using a universal simulator, we only show that the approximation error decays sup-exponentially fast. Furthermore, we also generalize these results to simulation from Markov processes, and simulation of random elements (or general random variables).