Estimating the Arc Length of the Optimal ROC Curve and Lower Bounding the Maximal AUC

We show that when the data likelihood ratio is used as the score function, the arc length of the corresponding ROC curve gives rise to a novel $f$-divergence which measures differences between the positive and negative data distributions. This $f$-divergence can be expressed using a variational objective and estimated only using samples from the positive and negative \emph{data} distributions. We show the empirical version of this variational objective is also a consistent estimator for the arctangent likelihood ratio with a non-parametric convergence rate $O_p(n^{-\beta/4})$ ($\beta \in (0,1]$ depends on the smoothness). Moreover, we show the surface area below the optimal ROC curve can be expressed as a similar variational objective depending on the arctangent likelihood ratio. These new insights lead to a novel two-step procedure for finding a good score function by lower bounding the maximal AUC. Experiments on CIFAR-10 datasets show the proposed two-step procedure achieves good AUC performance in imbalanced binary classification tasks while being less computationally demanding.