Local Linear Regression on Manifolds and its Geometric Interpretation

We study nonparametric regression with high-dimensional data, when the predictors lie on an unknown, lower-dimensional manifold. In this context, recently \cite{aswani_bickel:2011} suggested performing the conventional local linear regression (LLR) in the ambient space and regularizing the estimation problem using information obtained from learning the manifold locally. By contrast, our approach is to reduce the dimensionality first and then construct the LLR directly on a tangent plane approximation to the manifold. Under mild conditions, asymptotic expressions for the conditional mean squared error of the proposed estimator are derived for both the interior and the boundary cases. One implication of these results is that the optimal convergence rate depends only on the intrinsic dimension $d$ of the manifold, but not on the ambient space dimension $p$. Another implication is that the estimator is design adaptive and automatically adapts to the boundary of the unknown manifold. The bias and variance expressions are used to construct a simple and effective bandwidth selection rule. An extensive simulation study and an example are used to compare the computational speed and estimation accuracy of our method with that of those in \cite{aswani_bickel:2011}. The proposed method also has strong connection with manifold learning which is briefly discussed.