Fairly Accurate: Optimizing Accuracy Parity in Fair Target-Group Detection
<|reference_start|>Fairly Accurate: Optimizing Accuracy Parity in Fair Target-Group Detection: In algorithmic toxicity detection pipelines, it is important to identify which demographic group(s) are the subject of a post, a task commonly known as \textit{target (group) detection}. While accurate detection is clearly important, we further advocate a fairness objective: to provide equal protection to all groups who may be targeted. To this end, we adopt \textit{Accuracy Parity} (AP) -- balanced detection accuracy across groups -- as our fairness objective. However, in order to align model training with our AP fairness objective, we require an equivalent loss function. Moreover, for gradient-based models such as neural networks, this loss function needs to be differentiable. Because no such loss function exists today for AP, we propose \emph{Group Accuracy Parity} (GAP): the first differentiable loss function having a one-on-one mapping to AP. We empirically show that GAP addresses disparate impact on groups for target detection. Furthermore, because a single post often targets multiple groups in practice, we also provide a mathematical extension of GAP to larger multi-group settings, something typically requiring heuristics in prior work. Our findings show that by optimizing AP, GAP better mitigates bias in comparison with other commonly employed loss functions.<|reference_end|>
@article{gupta2024fairly,
    title={Fairly Accurate: Optimizing Accuracy Parity in Fair Target-Group
  Detection},
    author={Soumyajit Gupta, Venelin Kovatchev, Maria De-Arteaga, Matthew Lease},
    journal={arXiv preprint arXiv:2407.11933},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.11933},
    primaryClass={cs.LG}
}