Unbiased Risk Estimation in the Normal Means Problem via Coupled Bootstrap Techniques

We study a new method for estimating the risk of an arbitrary estimator of the mean vector in the classical normal means problem. The key idea is to generate two auxiliary data vectors, by adding carefully constructed normal noise vectors to the original data. We then train the estimator of interest on the first auxiliary vector and test it on the second. In order to stabilize risk estimate, we average this procedure over multiple draws of the synthetic noise. A key aspect of this coupled bootstrap approach is that it delivers an unbiased estimate of risk under no assumptions on the estimator of the mean vector, albeit for a slightly "harder" version of the original problem, where the noise variance is inflated. We show that, under the assumptions required for Stein's unbiased risk estimator (SURE), a limiting version of this estimator recovers SURE exactly. We also analyze a bias-variance decomposition of the error of our risk estimator, to elucidate the effects of the variance of the auxiliary noise and the number of bootstrap samples on the accuracy of the estimator. Lastly, we demonstrate that our coupled bootstrap risk estimator performs quite favorably in simulated experiments and in a denoising example.