Utility-Fairness Trade-Offs and How to Find Them
<|reference_start|>Utility-Fairness Trade-Offs and How to Find Them: When building classification systems with demographic fairness considerations, there are two objectives to satisfy: 1) maximizing utility for the specific task and 2) ensuring fairness w.r.t. a known demographic attribute. These objectives often compete, so optimizing both can lead to a trade-off between utility and fairness. While existing works acknowledge the trade-offs and study their limits, two questions remain unanswered: 1) What are the optimal trade-offs between utility and fairness? and 2) How can we numerically quantify these trade-offs from data for a desired prediction task and demographic attribute of interest? This paper addresses these questions. We introduce two utility-fairness trade-offs: the Data-Space and Label-Space Trade-off. The trade-offs reveal three regions within the utility-fairness plane, delineating what is fully and partially possible and impossible. We propose U-FaTE, a method to numerically quantify the trade-offs for a given prediction task and group fairness definition from data samples. Based on the trade-offs, we introduce a new scheme for evaluating representations. An extensive evaluation of fair representation learning methods and representations from over 1000 pre-trained models revealed that most current approaches are far from the estimated and achievable fairness-utility trade-offs across multiple datasets and prediction tasks.<|reference_end|>
@article{dehdashtian2024utility-fairness,
    title={Utility-Fairness Trade-Offs and How to Find Them},
    author={Sepehr Dehdashtian, Bashir Sadeghi, Vishnu Naresh Boddeti},
    journal={arXiv preprint arXiv:2404.09454},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.09454},
    primaryClass={cs.CV cs.CY cs.LG}
}