  Face biometrics are playing a key role in making modern smart city
applications more secure and usable. Commonly, the recognition threshold of a
face recognition system is adjusted based on the degree of security for the
considered use case. The likelihood of a match can be for instance decreased by
setting a high threshold in case of a payment transaction verification. Prior
work in face recognition has unfortunately showed that error rates are usually
higher for certain demographic groups. These disparities have hence brought
into question the fairness of systems empowered with face biometrics. In this
paper, we investigate the extent to which disparities among demographic groups
change under different security levels. Our analysis includes ten face
recognition models, three security thresholds, and six demographic groups based
on gender and ethnicity. Experiments show that the higher the security of the
system is, the higher the disparities in usability among demographic groups
are. Compelling unfairness issues hence exist and urge countermeasures in
real-world high-stakes environments requiring severe security levels.

Computer Vision and Pattern Recognition