Equality of Opportunity in Supervised Learning
We propose a criterion for discrimination against a specified sensitive attribute in supervised learning, where the goal is to predict some target based on available features. Assuming data about the predictor, target, and membership in the protected group are available, we show how to optimally adjust any learned predictor so as to remove discrimination according to our definition. Our framework also improves incentives by shifting the cost of poor classification from disadvantaged groups to the decision maker, who can respond by improving the classification accuracy. We enourage readers to consult the more complete manuscript on the arXiv.
Fairness in prediction is a problem that is becoming more and more acute nowadays. Suppose we are looking to find some classifier Z which we want to be fair towards a "protected" attribute A (e.g., race). The naive approach is to require that given  Pr[Z=1 | A=a] is the same for all values that A take, so that from knowing the classification of a datapoint leads to no advantage in guessing the point's attribute. This paper introduces an alternative notion of fairness. Given a sensitive attribute an a true label Y, then the classifier "equalizes odds" if for any value of y we have that Pr[Z=1 | Y=y, A=a] = Pr[Z=1 | Y=y, A=a'] for any two values a, a' of A. (It "equalizes opportunity" if the above holds solely for Y=1.) Hence, knowing the prediction gives no more advantage in inferring the sensitive attribute when one also knows the outcome Y.  The paper then goes to proving that post-processing a Z into a Z' that satisfies either property is feasible in poly-time, given the contingency table of (Z, Y, A), for binary predictors and also for predictors Z that use a "score" in the range [0,1] to classify. They also bound the "price" of fairness in the base of Bayesian classification and finally they test their approach on a FICO data of credit scores and the loaners who actually defaulted on their loans, in partition to race. None-surprisingly, the equal-odds classifier does slightly poorly than the non-fair classifier, but clearly outperforms the classifiers that preserve the naive notion of fairness.
This is a highly novel approach for fairness that is likely to have a great impact on machine learning. The paper is extremely interesting, very novel, has some nice theoretical results and interesting experiment, and moreover - it leads to *MANY* follow-up questions. I thus recommend it for full oral presentation.  My only complaint regarding the paper is regarding its style and presentation. I feel that a more detailed version (in the supplementary material section) with clarifications and *better figures* would have resulted in a much clearer paper. I had hard time following the proofs (which aren't wrong, just require some non-trivial deductions and keeping track of notation), and clearly, a longer pass at the paper would have contributed greatly to a better understanding. I urge the authors to do so, as I think this paper is clearly "one for the books" and I suspect it will be studied in depth in the future.  A few more specific styling comments: * Equation 3.1: "Similarly the second component is the false negative rate" --- isn't it the true positive rate? Isn't false negative rate be the second coordinate of \gamma_a(1-\hat Y)? * Lemma 3.2 (nitpicking) use parentheses for second coordinate $(\gamma_i(\hat Y))_2$  * Lemma 3.2, below: a bit more discussion on this polytope wouldn't hurt. In particular -- why (0,0) & (1,1) are there? (They are the "demographic classifiers", but why are they required? I couldn't see it from the proof of 3.3 in A.1) If \hat Y is multi-labeled, would you put all \gamma_a(Y=y) points? * Figure 1, legend: aren't the equal opportunity points w.r.t to \tilde Y? (i.e., it's \tilde Y=0 and \tilde Y = 1?) If not, then I really have missed something at your explanation and/or in the understanding of the figure... * Figure 2 completely confused me. Its explanation is baffling, and it seems unrelated to any of the text that actually appears in Section 3.2. Consider replacing it or putting it in the appendix with *MUCH* more instructions as to how the reader should interpret it. * The curve C_a(t) --- seems like, from the figure, it is (Pr[R < t | ...], Pr[R < t | ...]), or I just confused because you drew in Section 5 the CDFs? * Equation (3.3) unclear what you mean by convex-hull - especially since Figure 2 has the entire area below the curve highlighted, but then you say in line 176 that you ignore points below the main diagonal. (Am I right in understanding they are "bad than a random guess" because such points give pts with Y=0 better probability of getting classified as positive than pts with Y=1? This too, which seems simple, required me to parse through some definitions - which would have been far easier if you left room for more discussion...) * Figure 3 contributes nothing to the discussion. I suggest removing it. * Proof of 4.3: why not run the conversion proving (2) for every Y\in C without changing the classification cost, hence the conversion also holds for the best Y? * Figure 6: hard to parse. What is "Raw" and what is "within group"? * Proof of Proposition 3.4: Why not simply write the LP itself? Moreover, the proof gives the impression you just need variables for Pr[\tilde Y=\hat Y], but without dependence on A you cannot write the constraint \gamma_0(\tilde Y)=\gamma_1(\tilde Y).  * Proof of Thm 4.5, line 395 --- it is not easy for me. I had a hard time following... This is an example where pic would be useful...