Common fairness definitions in machine learning focus on balancing various notions of disparity and utility. In this work we study fairness in the context of risk disparity among sub-populations. We introduce the framework of Pareto-optimal fairness, where the goal of reducing risk disparity gaps is secondary only to the principle of not doing unnecessary harm, a concept that is especially applicable to high-stakes domains such as healthcare. We provide analysis and methodology to obtain maximally-fair no-unnecessary-harm classifiers on finite datasets. We argue that even in domains where fairness at cost is required, no-unnecesary-harm fairness can prove to be the optimal first step. This same methodology can also be applied to any unbalanced classification task, where we want to dynamically equalize the misclassification risks across outcomes without degrading overall performance any more than strictly necessary. We test the proposed methodology on real case-studies of predicting income, ICU patient mortality, classifying skin lesions from images, and assessing credit risk, demonstrating how the proposed framework compares favorably to other traditional approaches.
This paper considers the notion of "no-harm" group fairness, i.e. trying to reduce the risk gap between minority and majority groups without excessive reduction in performance on the majority groups. Authors formalize the problem by defining a Pareto fair classifier, i.e. one that minimizes the risk gaps between groups and belongs to the family of Pareto classifiers containing the classifier minimizing the empirical risk. Authors suggest an optimization procedure for finding the Pareto fair classifier and demonstrate its performance on multiple datasets.
Pros:
I think that studying "no-harm" classifiers is an important topic given the alarming tendency of some of the recent group fairness approaches to achieve fairness by essentially driving down the performance on the majority group without improving on the minority group. Decision making in medical applications is one of the prominent examples where "no-harm" is absolutely needed, as authors suggested. The mathematical formulation of the problem around the notion of Pareto optimality also seems reasonable.
Cons:
My concerns are related to counter-intuitive experimental results and lack of clarity in parts of the presentation.
Figure 1 seems important for understanding the ideas in the paper, but is not explained in much detail. Analogous to it Figure 2 is lacking important details. In the upper left plot, what are the decision boundaries of the baselines? What are the baselines risks in the center top figure, particularly for the equal risk classifier? It is hard to see from the right figure if the proposed classifier achieves the "no-harm" fairness over the equal risk classifier - numerical summary in a table could help. Finally, why is it necessary to use a neural network (which seems to be the case based on the supplement A.3) for the toy problem? I would recommend working through a toy example in more detail using a linear classifier to verify the correctness of the proposed technique and improve the overall clarity. Further, absence of a toy problem with linear classifier is alarming given there is not much discussion of the algorithm and its convergence properties.
Regarding the real data experiments, none seem to showcase the "no-harm" versus "zero-gap" fairness tradeoff motivating this paper.
MIMIC-III results seem to contradict the main story of the paper. The minority group appears to be "D/A/NW", then there should be a "harming" group fairness classifier achieving close to 0 discrimination at the cost of lowering performance on other subgroups. The "no-harm" classifier should then achieve a similar or slightly lower performance on "D/A/NW", but much better results on other sub-groups. Despite, the "no-harm" classifiers seems to outperform other approaches on "D/A/NW" by a good margin. Next, it appears that "Naive+Zafar" (it also would be helpful to have a brief discussion of the baselines considered) approach was not configured to eliminate A/S disparity as suggested by poor results on the D/A/NW and D/A/W, while it performs very well on other subpopulations.
Skin Lesion classification experiment departs from the problem of fairness and considers the problem of classification with unbalanced classes instead. Results are again counterintuitive. "Rebalanced Naive" only mildly improves over the "Naive" approach, while proposed algorithm seems to achieve a quite significant mean accuracy improvement. This again does not show the motivating "harm" vs "zero gap" tradeoff, but it could be interesting as the imbalanced classification is an important problem by itself. Could you please compare to more advanced imbalanced classification algorithms?
Results on the Adult and German Credit datasets are very similar across competing methods.
Acknowledgement and references to the individual fairness line of works are missing when presenting the problem of fairness in machine learning.
Font size in tables and Figure 2 legends is too small.
Typo in the last sentence on page 8: "have highly impactful" -> "are highly impactful".