title: MetaFaith: Faithful Natural Language Uncertainty Expression in LLMs
summary: A critical component in the trustworthiness of LLMs is reliable uncertainty
  communication, yet LLMs often use assertive language when conveying false
  claims, leading to over-reliance and eroded trust. We present the first
  systematic study of faithful confidence calibration of LLMs,
  benchmarking models' ability to use linguistic expressions of uncertainty that
  faithfully reflect their intrinsic uncertainty, across a
  comprehensive array of models, datasets, and prompting strategies. Our results
  demonstrate that LLMs largely fail at this task, and that existing
  interventions are insufficient: standard prompt approaches provide only
  marginal gains, and existing, factuality-based calibration techniques can even
  harm faithful calibration. To address this critical gap, we introduce
  MetaFaith, a novel prompt-based calibration approach inspired by human
  metacognition. We show that MetaFaith robustly improves faithful calibration
  across diverse models and task domains, enabling up to 61% improvement in
  faithfulness and achieving an 83% win rate over original generations as judged
  by humans.
paper_authors._id: 683d2a3651706d12b2cc8ace
paper_authors.name: Gabrielle Kaili-May Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-03T08:50:00.578Z
paper_authors._id: 65204db5b0e0d57453cb1809
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/endQHYuI6SuIWFkhTLjoS.png
paper_authors.fullname: Gabrielle Kaili-May Liu
paper_authors.type: user
paper_authors.user: pybeebee
paper_authors._id: 683d2a3651706d12b2cc8acf
paper_authors.name: Gal Yona
paper_authors._id: 683d2a3651706d12b2cc8ad0
paper_authors.name: Avi Caciularu
paper_authors._id: 683d2a3651706d12b2cc8ad1
paper_authors.name: Idan Szpektor
paper_authors._id: 683d2a3651706d12b2cc8ad2
paper_authors.name: Tim G. J. Rudner
paper_authors._id: 683d2a3651706d12b2cc8ad3
paper_authors.name: Arman Cohan
paper_title: MetaFaith: Faithful Natural Language Uncertainty Expression in LLMs
paper_summary: A critical component in the trustworthiness of LLMs is reliable uncertainty
  communication, yet LLMs often use assertive language when conveying false
  claims, leading to over-reliance and eroded trust. We present the first
  systematic study of faithful confidence calibration of LLMs,
  benchmarking models' ability to use linguistic expressions of uncertainty that
  faithfully reflect their intrinsic uncertainty, across a
  comprehensive array of models, datasets, and prompting strategies. Our results
  demonstrate that LLMs largely fail at this task, and that existing
  interventions are insufficient: standard prompt approaches provide only
  marginal gains, and existing, factuality-based calibration techniques can even
  harm faithful calibration. To address this critical gap, we introduce
  MetaFaith, a novel prompt-based calibration approach inspired by human
  metacognition. We show that MetaFaith robustly improves faithful calibration
  across diverse models and task domains, enabling up to 61% improvement in
  faithfulness and achieving an 83% win rate over original generations as judged
  by humans.
paper_ai_summary: A study reveals that Large Language Models (LLMs) struggle with expressing uncertainty accurately and introduces MetaFaith, a prompt-based method that enhances their calibration significantly.
paper_ai_keywords: faithful confidence calibration linguistic expressions of uncertainty intrinsic uncertainty prompting strategies metacognition