16
17

Nonparametric approximation of conditional expectation operators

Abstract

Given the joint distribution of two random variables X,YX,Y on some second countable locally compact Hausdorff space, we investigate the statistical approximation of the L2L^2-operator defined by [Pf](x):=E[f(Y)X=x][Pf](x) := \mathbb{E}[ f(Y) \mid X = x ] under minimal assumptions. By modifying its domain, we prove that PP can be arbitrarily well approximated in operator norm by Hilbert-Schmidt operators acting on a reproducing kernel Hilbert space. This fact allows to estimate PP uniformly by finite-rank operators over a dense subspace even when PP is not compact. In terms of modes of convergence, we thereby obtain the superiority of kernel-based techniques over classically used parametric projection approaches such as Galerkin methods. This also provides a novel perspective on which limiting object the nonparametric estimate of PP converges to. As an application, we show that these results are particularly important for a large family of spectral analysis techniques for Markov transition operators. Our investigation also gives a new asymptotic perspective on the so-called kernel conditional mean embedding, which is the theoretical foundation of a wide variety of techniques in kernel-based nonparametric inference.

View on arXiv
Comments on this paper