Trust-and-Verify Error Bounds for K-Nearest Neighbor Classifiers

We show that $k$-nearest neighbor classifiers, in spite of their famously fractured decision boundaries, have exponential error bounds with nearly O($n^{-\frac{1}{2}}$) Gaussian-style bound ranges, similar to error bounds based on VC dimension for other types of classifiers that have simpler decision boundaries. Specifically, we present an exponential PAC error bound for $k$-nearest neighbor classifiers that has O($n^{-\frac{1}{2}}\sqrt{(k + \ln n)(\ln \ln n + \frac{1}{\delta})}$) error bound range, for $n$ in-sample examples and bound failure probability $\delta$.