A Comparison of Differential Performance Metrics for the Evaluation of Automatic Speaker Verification Fairness
<|reference_start|>A Comparison of Differential Performance Metrics for the Evaluation of Automatic Speaker Verification Fairness: When decisions are made and when personal data is treated by automated processes, there is an expectation of fairness -- that members of different demographic groups receive equitable treatment. This expectation applies to biometric systems such as automatic speaker verification (ASV). We present a comparison of three candidate fairness metrics and extend previous work performed for face recognition, by examining differential performance across a range of different ASV operating points. Results show that the Gini Aggregation Rate for Biometric Equitability (GARBE) is the only one which meets three functional fairness measure criteria. Furthermore, a comprehensive evaluation of the fairness and verification performance of five state-of-the-art ASV systems is also presented. Our findings reveal a nuanced trade-off between fairness and verification accuracy underscoring the complex interplay between system design, demographic inclusiveness, and verification reliability.<|reference_end|>
@article{chouchane2024a,
    title={A Comparison of Differential Performance Metrics for the Evaluation of
  Automatic Speaker Verification Fairness},
    author={Oubaida Chouchane, Christoph Busch, Chiara Galdi, Nicholas Evans,
  Massimiliano Todisco},
    journal={arXiv preprint arXiv:2404.17810},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.17810},
    primaryClass={eess.AS cs.SD}
}