Domain-adaptive and Subgroup-specific Cascaded Temperature Regression for Out-of-distribution Calibration
<|reference_start|>Domain-adaptive and Subgroup-specific Cascaded Temperature Regression for Out-of-distribution Calibration: Although deep neural networks yield high classification accuracy given sufficient training data, their predictions are typically overconfident or under-confident, i.e., the prediction confidences cannot truly reflect the accuracy. Post-hoc calibration tackles this problem by calibrating the prediction confidences without re-training the classification model. However, current approaches assume congruence between test and validation data distributions, limiting their applicability to out-of-distribution scenarios. To this end, we propose a novel meta-set-based cascaded temperature regression method for post-hoc calibration. Our method tailors fine-grained scaling functions to distinct test sets by simulating various domain shifts through data augmentation on the validation set. We partition each meta-set into subgroups based on predicted category and confidence level, capturing diverse uncertainties. A regression network is then trained to derive category-specific and confidence-level-specific scaling, achieving calibration across meta-sets. Extensive experimental results on MNIST, CIFAR-10, and TinyImageNet demonstrate the effectiveness of the proposed method.<|reference_end|>
@article{wang2024domain-adaptive,
    title={Domain-adaptive and Subgroup-specific Cascaded Temperature Regression
  for Out-of-distribution Calibration},
    author={Jiexin Wang, Jiahao Chen, Bing Su},
    journal={2024 IEEE International Conference on Acoustics, Speech, and
  Signal Processing (ICASSP 2024), Seoul, Korea},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.09204},
    primaryClass={cs.CV}
}