A Theoretical Framework for Bayesian Nonparametric Regression: Orthonormal Random Series and Rates of Contraction

We develop a unifying framework for Bayesian nonparametric regression to study the rates of contraction with respect to the integrated $L_2$-distance without assuming the regression function space to be uniformly bounded. The framework is built upon orthonormal random series in a flexible manner. A general theorem for deriving rates of contraction for Bayesian nonparametric regression is provided under the proposed framework. Three non-trivial applications of the proposed framework are provided: The finite random series regression of an $\alpha$-H\"older function, with adaptive rates of contraction up to a logarithmic factor, given independent and uniform design points; The un-modified block prior regression of an $\alpha$-Sobolev function, with adaptive-and-exact rates of contraction, given independent and uniform design points; The squared-exponential Gaussian process regression of a supersmooth function with a near-parametric rate of contraction, under the condition that the design points are fixed and reasonably spread. These applications serve as generalization or complement of their respective results in the literature. Extension to sparse additive models in high dimensions is discussed as well.