title: Reasoning Models Better Express Their Confidence
summary: Despite their strengths, large language models (LLMs) often fail to
  communicate their confidence accurately, making it difficult to assess when
  they might be wrong and limiting their reliability. In this work, we
  demonstrate that reasoning models-LLMs that engage in extended chain-of-thought
  (CoT) reasoning-exhibit superior performance not only in problem-solving but
  also in accurately expressing their confidence. Specifically, we benchmark six
  reasoning models across six datasets and find that they achieve strictly better
  confidence calibration than their non-reasoning counterparts in 33 out of the
  36 settings. Our detailed analysis reveals that these gains in calibration stem
  from the slow thinking behaviors of reasoning models-such as exploring
  alternative approaches and backtracking-which enable them to adjust their
  confidence dynamically throughout their CoT, making it progressively more
  accurate. In particular, we find that reasoning models become increasingly
  better calibrated as their CoT unfolds, a trend not observed in non-reasoning
  models. Moreover, removing slow thinking behaviors from the CoT leads to a
  significant drop in calibration. Lastly, we show that these gains are not
  exclusive to reasoning models-non-reasoning models also benefit when guided to
  perform slow thinking via in-context learning.
paper_authors._id: 682d55ecea67e90811b09b6b
paper_authors.name: Dongkeun Yoon
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-21T08:40:23.897Z
paper_authors._id: 617f679fb15f8a665f3999fc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/617f679fb15f8a665f3999fc/NW1vkLsGAlWpAQYTux05X.jpeg
paper_authors.fullname: Dongkeun Yoon
paper_authors.type: user
paper_authors.user: DKYoon
paper_authors._id: 682d55ecea67e90811b09b6c
paper_authors.name: Seungone Kim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:15:44.937Z
paper_authors._id: 6469949654873f0043b09c22
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6469949654873f0043b09c22/Lk7IJAR16Wa_sGJ2g81AQ.jpeg
paper_authors.fullname: Seungone Kim
paper_authors.type: user
paper_authors.user: seungone
paper_authors._id: 682d55ecea67e90811b09b6d
paper_authors.name: Sohee Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:15:51.388Z
paper_authors._id: 606ae0adb579bb0e88515311
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1617617060380-noauth.jpeg
paper_authors.fullname: Sohee Yang
paper_authors.type: user
paper_authors.user: soheeyang
paper_authors._id: 682d55ecea67e90811b09b6e
paper_authors.name: Sunkyoung Kim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:16:00.139Z
paper_authors._id: 628efeff6d8dcc7ab5263881
paper_authors.avatarUrl: /avatars/1e894ccd7ba206a755b0b9af9f22ead1.svg
paper_authors.fullname: Sunkyoung Kim
paper_authors.type: user
paper_authors.user: Sunkyoung
paper_authors._id: 682d55ecea67e90811b09b6f
paper_authors.name: Soyeon Kim
paper_authors._id: 682d55ecea67e90811b09b70
paper_authors.name: Yongil Kim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:17:46.138Z
paper_authors._id: 66a9c79cf59bc4c77baac0d5
paper_authors.avatarUrl: /avatars/57e9943b339a5b9be21b411e5286821c.svg
paper_authors.fullname: Yongil Kim
paper_authors.type: user
paper_authors.user: YongilKim
paper_authors._id: 682d55ecea67e90811b09b71
paper_authors.name: Eunbi Choi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:17:29.667Z
paper_authors._id: 6044fd39e6aa3e130cb92867
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6044fd39e6aa3e130cb92867/L5hb8vpHY6SKMEL-Xacma.jpeg
paper_authors.fullname: Eunbi Choi
paper_authors.type: user
paper_authors.user: unbiarirang
paper_authors._id: 682d55ecea67e90811b09b72
paper_authors.name: Yireun Kim
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-05-21T04:27:50.114Z
paper_authors._id: 660260cf1737e5cd4a826550
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/660260cf1737e5cd4a826550/AlSfoM2WtqPjLtYR6x7Wf.jpeg
paper_authors.fullname: Yireun Kim
paper_authors.type: user
paper_authors.user: yireun
paper_authors._id: 682d55ecea67e90811b09b73
paper_authors.name: Minjoon Seo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:17:22.628Z
paper_authors._id: 621f05ba970615ad5861ceb1
paper_authors.avatarUrl: /avatars/7e1902aa71369a524afda9b0a9e88e22.svg
paper_authors.fullname: Minjoon Seo
paper_authors.type: user
paper_authors.user: minjoon
paper_title: Reasoning Models Better Express Their Confidence
paper_summary: Despite their strengths, large language models (LLMs) often fail to
  communicate their confidence accurately, making it difficult to assess when
  they might be wrong and limiting their reliability. In this work, we
  demonstrate that reasoning models-LLMs that engage in extended chain-of-thought
  (CoT) reasoning-exhibit superior performance not only in problem-solving but
  also in accurately expressing their confidence. Specifically, we benchmark six
  reasoning models across six datasets and find that they achieve strictly better
  confidence calibration than their non-reasoning counterparts in 33 out of the
  36 settings. Our detailed analysis reveals that these gains in calibration stem
  from the slow thinking behaviors of reasoning models-such as exploring
  alternative approaches and backtracking-which enable them to adjust their
  confidence dynamically throughout their CoT, making it progressively more
  accurate. In particular, we find that reasoning models become increasingly
  better calibrated as their CoT unfolds, a trend not observed in non-reasoning
  models. Moreover, removing slow thinking behaviors from the CoT leads to a
  significant drop in calibration. Lastly, we show that these gains are not
  exclusive to reasoning models-non-reasoning models also benefit when guided to
  perform slow thinking via in-context learning.
paper_ai_summary: Reasoning models, which perform chain-of-thought reasoning, exhibit superior confidence calibration compared to non-reasoning models by dynamically adjusting their confidence during the reasoning process.
paper_ai_keywords: large language models LLMs chain-of-thought CoT reasoning confidence calibration benchmarking datasets alternative approaches backtracking in-context learning