Equalized odds

Content sourced from Wikipedia, licensed under CC BY-SA 3.0.

Equalized odds (also called conditional procedure accuracy equality or disparate mistreatment) is a fairness criterion for machine learning classifiers. It requires that a model’s error rates are the same across protected and unprotected groups.

Specifically, for any true outcome Y (whether a person actually qualifies) and group A, the probability that the model outputs R = + should not depend on the group. That is, P(R = + | Y = y, A = a) = P(R = + | Y = y, A = b) for all y in {+, -} and all groups a, b. In practice, this means the true positive rate (correctly predicting + when Y = +) and the false positive rate (predicting + when Y = -) are equal across groups.

Example: If A is gender or race and Y is whether a person is qualified for a degree, R is the school’s decision to offer the degree. Equalized odds says that among those who are actually qualified, the offer rate should be the same across groups, and among those not qualified, the offer rate should also be the same across groups. If base rates differ between groups, achieving equalized odds may still result in different overall enrollment rates to keep the conditional rates equal.

Originally defined for binary Y, equalized odds was extended to multiple classes in 2017.