We propose an approach to training machine learning models that are fair in the sense that their performance is invariant under certain perturbations to the features. For example, the performance of a resume screening system should be invariant under changes to the name of the applicant. We formalize this intuitive notion of fairness by connecting it to the original notion of individual fairness put forth by Dwork et al and show that the proposed approach achieves this notion of fairness. We also demonstrate the effectiveness of the approach on two machine learning tasks that are susceptible to gender and racial biases.
This paper proposes a new definition of algorithmic fairness that is based on the idea of individual fairness. They then present an algorithm that will provably find an ML model that satisfies the fairness constraint (if such a model exists in the search space). One needed ingredient for the fairness constraint is a distance function (or "metric") in the input space that captures the fact that some features should be irrelevant to the classification task. That is, under this distance function, input that differ only in sensitive attributes like race or gender should be close-by. The idea of the fairness constraint is that by perturbing the inputs (while keeping them close with respect to the distance function), the loss of the model cannot be significantly increased. Thus, this fairness constraint is very much related to robustness.
---
Overall, I like the basic idea of the paper but I found the presentation lacking.
I do think their idea for a fairness constraint is very interesting, but it gets too bogged down in the details of the mathematical theory. They mention Dwork et al. at the beginning but don't really compare it to their idea in detail, even though I think there would be a lot of interesting things to say about this. For example, the definition by Dwork et al. seems to imply that some labels in the training set might be incorrect, whereas the definition in this paper does not seem to imply that (which I think is a good thing).
The main problem in section 2 is that the choice of distance function is barely discussed although that's what's most important to make the result fair. For all the mathematical rigor in section 2, the paragraph that is arguing that the defined constraint encourages fairness is somewhat weak. Here a comparison to other fairness definitions and an in-depth discussion of the distance function would help.
(In general I felt that this part was more trying to impress the reader than trying to explain, but I will try to not hold it against this paper.)
As it is, I feel the paper cannot be completely understood without reading the appendix.
There is also this sentence at the bottom of page 5: "A small gap implies the investigator cannot significantly increase the loss by moving samples from $P_*$ to comparable samples." This should have been at the beginning of section 2 in order to motivate the derivation.
In the experiments, I'm not sure how useful the result of the word embedding experiment really is. Either someone is interested in the sentiment associated with names, in which case your method renders the predicted sentiments useless or someone is not interested in the sentiment associated with names and your method doesn't even have any effect.
Final point: while I like the idea of the balanced TPR, I think the name is a bit misleading because, for example, in the binary case it is the average of the TPR and the TNR. Did you invent this terminology? If so, might I suggest another name like balanced accuracy?
I would change the score (upwards) if the following things are addressed:
- make it easier to understand the main point of the paper
- make more of a comparison to Dwork et al. or other fairness definitions
- fix the following minor mistakes
Minor comments:
- page 2, beginning of section 2: you use the word "regulator" here once but everywhere else you use "investigator"
- equation 2.1: as far as I can tell $M$ is not defined anywhere; you might mean $\Delta (\mathcal{Z})$
- page 3, sentence before Eq 2.3: what does the $\#$ symbol mean?
- page 3, sentence before Eq 2.3: what is $T$? is it $T_\lambda$?
- Algorithm 2: what is the difference between $\lambda^*_t$ and $\hat{\lambda}_t$?
- page 7: you used a backslash between "90%" and "10%" and "train" and "test". That would traditionally be a normal slash.
- in appendix B: the explanation for what $P_{ran(A)}$ means should be closer to the first usage
- in the references, you list one paper twice (the one by Zhang et al.)
EDIT: changed the score after looking at the revised version