Robust performance metrics for imbalanced classification problems
<|reference_start|>Robust performance metrics for imbalanced classification problems: We show that established performance metrics in binary classification, such as the F-score, the Jaccard similarity coefficient or Matthews' correlation coefficient (MCC), are not robust to class imbalance in the sense that if the proportion of the minority class tends to $0$, the true positive rate (TPR) of the Bayes classifier under these metrics tends to $0$ as well. Thus, in imbalanced classification problems, these metrics favour classifiers which ignore the minority class. To alleviate this issue we introduce robust modifications of the F-score and the MCC for which, even in strongly imbalanced settings, the TPR is bounded away from $0$. We numerically illustrate the behaviour of the various performance metrics in simulations as well as on a credit default data set. We also discuss connections to the ROC and precision-recall curves and give recommendations on how to combine their usage with performance metrics.<|reference_end|>
@article{holzmann2024robust,
    title={Robust performance metrics for imbalanced classification problems},
    author={Hajo Holzmann and Bernhard Klar},
    journal={arXiv preprint arXiv:2404.07661},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.07661},
    primaryClass={stat.ML cs.LG stat.ME}
}