EigenGP: KL-expansion based Gaussian process learning

Gaussian processes (GPs) provide a nonparametric representation of functions. Given N training points, the exact GP inference incurs high computational cost. In this paper, we propose a sparse Gaussian process model, EigenGP, based on Karhunen-Lo\`eve (KL) expansions of a GP prior. We use the Nystr\"{o}m approximation to obtain eigenfunctions of the covariance function and use an empirical Bayesian approach to select these eigenfunctions. To handle nonlinear likelihoods, we develop an efficient expectation propagation inference algorithm, and couple it with expectation maximization for evidence maximization. By selecting eigenfunctions of Gaussian kernels that are associated with data clusters, EigenGP is also suitable for semi-supervised learning. Our experimental results demonstrate improved predictive performance of EigenGP over alternative state-of-the-art sparse GP and semi-supervised learning methods for regression,supervised and semi-supervised classification.