title: When Do LLMs Admit Their Mistakes? Understanding the Role of Model
    Belief in Retraction
summary: Can large language models (LLMs) admit their mistakes when they should know
  better? In this work, we define the behavior of acknowledging errors in
  previously generated answers as "retraction" and aim to understand when and why
  LLMs choose to retract. We first construct model-specific datasets to evaluate
  whether a model will retract an incorrect answer that contradicts its own
  parametric knowledge. While LLMs are capable of retraction, they do so only
  infrequently. We demonstrate that retraction is closely tied to previously
  identified indicators of models' internal belief: models fail to retract wrong
  answers that they "believe" to be factually correct. Steering experiments
  further demonstrate that internal belief causally influences model retraction.
  In particular, when the model does not believe its answer, this not only
  encourages the model to attempt to verify the answer, but also alters attention
  behavior during self-verification. Finally, we demonstrate that simple
  supervised fine-tuning significantly improves retraction performance by helping
  the model learn more accurate internal beliefs. Code and datasets are available
  on https://github.com/ayyyq/llm-retraction.
paper_authors._id: 68301340cc0d12cd873342e1
paper_authors.name: Yuqing Yang
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-05-23T06:35:23.129Z
paper_authors._id: 65c0de12efbb14b39c97f78e
paper_authors.avatarUrl: /avatars/18485f79427a35bd9e19f71b67c88dce.svg
paper_authors.fullname: Yuqing Yang
paper_authors.type: user
paper_authors.user: ayyyq
paper_authors._id: 68301340cc0d12cd873342e2
paper_authors.name: Robin Jia
paper_title: When Do LLMs Admit Their Mistakes? Understanding the Role of Model
    Belief in Retraction
paper_summary: Can large language models (LLMs) admit their mistakes when they should know
  better? In this work, we define the behavior of acknowledging errors in
  previously generated answers as "retraction" and aim to understand when and why
  LLMs choose to retract. We first construct model-specific datasets to evaluate
  whether a model will retract an incorrect answer that contradicts its own
  parametric knowledge. While LLMs are capable of retraction, they do so only
  infrequently. We demonstrate that retraction is closely tied to previously
  identified indicators of models' internal belief: models fail to retract wrong
  answers that they "believe" to be factually correct. Steering experiments
  further demonstrate that internal belief causally influences model retraction.
  In particular, when the model does not believe its answer, this not only
  encourages the model to attempt to verify the answer, but also alters attention
  behavior during self-verification. Finally, we demonstrate that simple
  supervised fine-tuning significantly improves retraction performance by helping
  the model learn more accurate internal beliefs. Code and datasets are available
  on https://github.com/ayyyq/llm-retraction.
paper_ai_summary: LLMs rarely retract incorrect answers they believe to be factually correct, but supervised fine-tuning can improve their retraction performance by refining their internal beliefs.
paper_ai_keywords: retraction model-specific datasets parametric knowledge internal belief self-verification attention behavior supervised fine-tuning