title: Measuring Calibration in Deep Learning
abstract: Overconfidence and underconfidence in machine learning classifiers is measured by calibration: the degree to which the probabilities predicted for each class match the accuracy of the classifier on that prediction. We propose two new measures for calibration, the Static Calibration Error (SCE) and Adaptive Calibration Error (ACE). These measures take into account every prediction made by a model, in contrast to the popular Expected Calibration Error.
authors: Jeremy Nixon Mike Dusenberry Ghassen Jerfel Linchuan Zhang Dustin Tran
hIndex: Dustin Tran 29