Concentration behavior of the penalized least squares estimator

Consider the standard nonparametric regression model with true function $f^0$ and let $\hat{f}$ be the penalized least squares estimator. Denote by $\tau(\hat{f})$ the trade-off between closeness to $f^0$ and complexity penalization of $\hat{f}$, where complexity is described by a seminorm on a class of functions. We first show that $\tau(\hat{f})$ is concentrated with high probability around a constant depending on the sample size. Then, under some conditions and for the proper choice of the smoothing parameter, we obtain bounds for this non-random quantity. This allow us to derive explicit upper and lower bounds for $\tau(\hat{f})$ that hold with high probability. We illustrate our results with some examples that include the smoothing splines estimator.