Uniform Concentration of the Loss Estimator for Neural DUDE

We give a theoretical justification of the concentration property observed for the recently developed neural network-based sliding window discrete denoiser, Neural DUDE. Namely, we rigorously prove that the estimated loss devised for Neural DUDE, computed solely from the noisy data, concentrates on the true denoising loss, which can only be evaluated with the underlying clean data. The concentration is shown to hold in a strong sense, i.e., it uniformly holds over \emph{all} the bounded network parameters including the parameters of Neural DUDE and over \emph{all} possible underlying clean data, with high probability. Moreover, we characterize the sufficient condition for the concentration, in terms of the sliding window size $k$ and the data size $n$, as $k=o(\sqrt{n})$, which is a much weaker condition than that of DUDE, the predecessor of Neural DUDE. For the proof, we make a novel application of the tools of the learning theory, e.g., Rademacher complexity. We conclude with experimental results that highlight the theoretical results and advocate the hyperparamter selection method of Neural DUDE.