EigenGP: Gaussian process models with adaptive eigenfunctions

Gaussian processes (GPs) provide a nonparametric representation of functions. However, classical GP inference suffers from high computational cost and it is difficult to design nonstationary GP priors in practice. In this paper, we propose a sparse Gaussian process model, EigenGP, based on data-dependent eigenfunctions of a GP prior. The data-dependent eigenfunctions make the Gaussian process nonstationary and can be viewed as dictionary elements--accordingly our algorithm conducts adaptive Bayesian dictionary learning. We learn all hyper- parameters including basis points and lengthscales from data by maximizing the model marginal likelihood; we explore computational linear algebra to simplify the gradient computation significantly. Our experimental results demonstrate improved predictive performance of EigenGP over alternative state-of-the-art sparse GP methods such as sparse spectrum Gaussian process regression.