Does mitigating ML's impact disparity require treatment disparity?
Following precedent in employment discrimination law, two notions of disparity are widely-discussed in papers on fairness and ML. Algorithms exhibit treatment disparity if they formally treat members of protected subgroups differently; algorithms exhibit impact disparity when outcomes differ across subgroups (even unintentionally). Naturally, we can achieve impact parity through purposeful treatment disparity. One line of papers aims to reconcile the two parities proposing disparate learning processes (DLPs). Here, the sensitive feature is used during training but a group-blind classifier is produced. In this paper, we show that: (i) when sensitive and (nominally) nonsensitive features are correlated, DLPs will indirectly implement treatment disparity, undermining the policy desiderata they are designed to address; (ii) when group membership is partly revealed by other features, DLPs induce within-class discrimination; and (iii) in general, DLPs provide suboptimal trade-offs between accuracy and impact parity. Experimental results on several real-world datasets highlight the practical consequences of applying DLPs.
Summary:  This paper describes a series of negative results for "disparate learning processes" (DLPs). These are recent methods to train classification models whose predictions obey a given fairness constraint between protected subgroups (e.g., by solving a convex optimization problem or preprocessing). The key characteristic of DLPs is that they have access to labels for the protected groups at training time, but not at deployment. In practice, this means that a DLP will produce a single model for the entire population.  In this paper, the authors present several arguments against DLPs, including:   1. If the protected attribute is redundantly encoded in other features, then a DLP may result in the same treatment disparity that aims to overcome.  2. If the protected attributed is partially encoded in other features, then a DLP may induce within class-discrimination and harm certain members of the protected group.  3. A DLP provides suboptimal accuracy in comparison to other classifiers that satisfy a specific family of fairness constraints.   The authors present several proofs of the theorems for an idealized setting. They complement the theorems with a series of empirical results on a real-world dataset and UCI datasets showing the shortcomings of DLPs and the superiority of simple alternatives (e.g. training a logistic regression model and applying per-group thresholding).  Comments:  I thought that this is a well-written paper on an interesting and timely topic. I think that it is important to state that the central contribution of this paper is NOT methodological novelty or theoretical novelty, but rather the results against DLPs. I believe that these results will be significant in the long-term as much recent work in this area has focused on tackling fairness-related issues through DLPs without considering their shortcomings.  I believe that the current paper would significantly strengthened if the authors were to consider broader approaches to treatment disparity. For example, many readers would consider Dwork et al's proposed approach to train “separate models for separate groups” as a natural extension to the per-group thresholding used in the numerical experiments. The authors currently cite these methods in the text under the “alternative approaches” in Section 2 and the “fairness beyond disparate impact” in Section 5. However, they should be discussed more thoroughly and included in the experimental results in Section 4.    Other than this, two other major points to address:  - Among the arguments presented against DLPs, 1. deserves some qualification.  It is unlikely that a practitioner would not check for that the protected attribute is redundantly encoded before training the model.  - The results in Section 3 apply to a setting where the authors have access to the optimal classifier d*. This is fine given that the results are in line with their empirical work in Section 4. However, the authors should explicitly state that this does not capture the real-world setting (i.e., where the classifier is learned from data). There should also be a discussion as to when can should expect the results to manifest themselves in the real-world setting  Specific Comments:  - The main findings (e.g. points 1. through 4) at the end of the Introduction are great. I would recommend adding a short example after each point so that readers from a broader audience will be able to understand the points without having to read the papers.  - Proofs in Section 3 could be put into an Appendix for the sake of space.  - The “Complicating matters…” (line 29) in the Introduction is interesting and merits further discussion. I’d suggest placing some of the discussion from lines 318 – 326 here.   - “Alternative Approach” in Section 2 and the “Fairness beyond Disparate Impact” in Section 5 should be merged (see also comments above)  - Section 2: 0-1 loss should be defined in line 70. I'd recommend changing the Section title to Preliminaries.  Minor Comments:  - Table 2: the formatting can be improved. The column labels should be in a different  - Line 6 “\textit{dis}parity”