A Possibility in Algorithmic Fairness: Calibrated Scores for Fair Classifications

Calibration and equal error rates are fundamental criteria of algorithmic fairness that have been shown to conflict with one another. This paper proves that they can be satisfied simultaneously in settings where decision-makers use risk scores to assign binary treatments. In particular, we derive necessary and sufficient conditions for the existence of calibrated scores that yield classifications achieving equal error rates. We then present an algorithm that searches for the most informative score subject to both calibration and minimal error rate disparity. Applied to a real criminal justice risk assessment, we show that our method can eliminate error disparities while maintaining calibration. In a separate application to credit lending, the procedure provides a solution that is more fair and profitable than a common alternative that omits sensitive features.