Self-Consistency Boosts Calibration for Math Reasoning
<|reference_start|>Self-Consistency Boosts Calibration for Math Reasoning: Calibration, which establishes the correlation between accuracy and model confidence, is important for LLM development. We design three off-the-shelf calibration methods based on self-consistency (Wang et al., 2022) for math reasoning tasks. Evaluation on two popular benchmarks (GSM8K and MathQA) using strong open-source LLMs (Mistral and LLaMA2), our methods better bridge model confidence and accuracy than existing methods based on p(True) (Kadavath et al., 2022) or logit (Kadavath et al., 2022).<|reference_end|>
@article{wang2024self-consistency,
    title={Self-Consistency Boosts Calibration for Math Reasoning},
    author={Ante Wang, Linfeng Song, Ye Tian, Baolin Peng, Lifeng Jin, Haitao Mi,
  Jinsong Su and Dong Yu},
    journal={arXiv preprint arXiv:2403.09849},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.09849},
    primaryClass={cs.CL cs.AI}
}