Recent work on Bayesian updating in infinite-dimensional parameter spaces has established conditions under which the posterior distribution will concentrate on the truth, if the latter has a perfect representation within the support of the prior, subject to dynamical restrictions such as independent or Markovian data. Here I establish sufficient conditions for the convergence of the posterior distribution in non-parametric problems when all hypotheses are wrong, and the data-generating process has a complicated dependence structure. The main dynamical assumption is the generalized asymptotic equipartition (or ``Shannon-McMillan-Breiman'') property of information theory, which is used in conjunction with Egorov's theorem on uniform convergence to construct a sieve and so control the fluctuations of the integrated log-likelihood. In addition to posterior convergence, I derive a kind of large deviations principle for the posterior measure, extending in some cases to rates of convergence, and discuss the advantages of predicting using a combination of models known to be wrong. An appendix sketches connections between the present results and the replicator dynamics of evolutionary theory.
View on arXiv