We consider the problem of fair and robust model training in the presence of data poisoning. Ensuring fairness usually involves a tradeoff against accuracy, so if the data poisoning is mistakenly viewed as additional bias to be fixed, the accuracy will be sacrificed even more. We demonstrate that this phenomenon indeed holds for state-of-the-art model fairness techniques. We then propose FR-GAN, which holistically performs fair and robust model training using generative adversarial networks (GANs). We first use a generator that attempts to classify examples as accurately as possible. In addition, we deploy two discriminators: (1) a fairness discriminator that predicts the sensitive attribute from classification results and (2) a robustness discriminator that distinguishes examples and predictions from a clean validation set. Our framework respects all the prominent fairness measures: disparate impact, equalized odds, and equal opportunity. Also, FR-GAN optimizes fairness without requiring the knowledge of prior statistics of the sensitive attributes. In our experiments, FR-GAN shows almost no decrease in fairness and accuracy in the presence of data poisoning unlike other state-of-the-art fairness methods, which are vulnerable. In addition, FR-GAN can be adjusted using parameters to maintain reasonable accuracy and fairness even if the validation set is too small or unavailable.
This paper proposes to use a GAN style approach for training a classifier that is robust to data poisoning and can achieve a pre-specified notion of group fairness.
The contribution of this paper is incremental in the context of prior Adversarial Debasing (AD) approach using essentially same GAN for group fairness and prior work presenting ideas of utilizing clean validation data to defend against data poisoning. This paper is proposing to add an additional discriminator to the AD approach that distinguishes training data and clean validation data. If the training data is poisoned, such distinguishment may be possible and maximizing loss of this discriminator can help to robustify against poisoned samples.
Experimental results are insufficient to argue improvement over the AD. There are no AD results in the equalized odds Adult experiment in the supplement. I recommend more detailed comparison against the AD method (including results showing confusion matrices). Also note that AD, as presented in the original paper, is optimizing for demographic parity, but can also be adjusted to other group fairness metrics. Finally, in the context of Adult dataset, it is important to also report performance metrics such as balanced TPR since the labels are quite imbalanced.
Are there any real data examples where poisoning does not need to be introduced artificially and the proposed method helps to improve the fairness properties? I think an interesting direction could be to consider data where labels are subjective. For example, a dataset on loan decisions can be naturally "poisoned" with human biases, i.e. information that someone did not receive the loan may be due to an error or bias of a human in charge of the decision making.
Lastly I think that the discussion of the prior related work on fairness is incomplete. This paper exclusively covers group fairness, which indeed has been shown to have some disadvantages. For example, prior work [1] has shown that some group fairness notions can not be satisfied simultaneously in certain cases. In this regard it is also important to report multiple group fairness metrics simultaneously in the experiments. The other fairness definition, that has not been mentioned in this paper, is individual fairness [2]. It has legal and intuitive interpretations. Multiple recent papers have explored the direction of individual fairness [3,4,5,6].
[1] Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016). Inherent trade-offs in the fair determination of risk scores.
[2] Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012, January). Fairness through awareness.
[3] Garg, S., Perot, V., Limtiaco, N., Taly, A., Chi, E. H., & Beutel, A. (2019, January). Counterfactual fairness in text classification through robustness.
[4] Yurochkin, M., Bower, A., & Sun, Y. (2019). Learning fair predictors with Sensitive Subspace Robustness.
[5] Kearns, M., Roth, A., & Sharifi-Malvajerdi, S. (2019). Average Individual Fairness: Algorithms, Generalization and Experiments.
[6] Jung, C., Kearns, M., Neel, S., Roth, A., Stapleton, L., & Wu, Z. S. (2019). Eliciting and Enforcing Subjective Individual Fairness.