Contexts Matter: An Empirical Study on Contextual Influence in Fairness Testing for Deep Learning Systems
<|reference_start|>Contexts Matter: An Empirical Study on Contextual Influence in Fairness Testing for Deep Learning Systems: Background: Fairness testing for deep learning systems has been becoming increasingly important. However, much work assumes perfect context and conditions from the other parts: well-tuned hyperparameters for accuracy; rectified bias in data, and mitigated bias in the labeling. Yet, these are often difficult to achieve in practice due to their resource-/labour-intensive nature. Aims: In this paper, we aim to understand how varying contexts affect fairness testing outcomes. Method:We conduct an extensive empirical study, which covers $10,800$ cases, to investigate how contexts can change the fairness testing result at the model level against the existing assumptions. We also study why the outcomes were observed from the lens of correlation/fitness landscape analysis. Results: Our results show that different context types and settings generally lead to a significant impact on the testing, which is mainly caused by the shifts of the fitness landscape under varying contexts. Conclusions: Our findings provide key insights for practitioners to evaluate the test generators and hint at future research directions.<|reference_end|>
@article{du2024contexts,
    title={Contexts Matter: An Empirical Study on Contextual Influence in Fairness
  Testing for Deep Learning Systems},
    author={Chengwen Du, Tao Chen},
    journal={arXiv preprint arXiv:2408.06102},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.06102},
    primaryClass={cs.SE cs.CY cs.LG}
}