When Do LLMs Admit Their Mistakes? Understanding the Role of Model
  Belief in Retraction
Can large language models (LLMs) admit their mistakes when they should know
better? In this work, we define the behavior of acknowledging errors in
previously generated answers as "retraction" and aim to understand when and why
LLMs choose to retract. We first construct model-specific datasets to evaluate
whether a model will retract an incorrect answer that contradicts its own
parametric knowledge. While LLMs are capable of retraction, they do so only
infrequently. We demonstrate that retraction is closely tied to previously
identified indicators of models' internal belief: models fail to retract wrong
answers that they "believe" to be factually correct. Steering experiments
further demonstrate that internal belief causally influences model retraction.
In particular, when the model does not believe its answer, this not only
encourages the model to attempt to verify the answer, but also alters attention
behavior during self-verification. Finally, we demonstrate that simple
supervised fine-tuning significantly improves retraction performance by helping
the model learn more accurate internal beliefs. Code and datasets are available
on https://github.com/ayyyq/llm-retraction.
68301340cc0d12cd873342e1
Yuqing Yang
extracted_confirmed
2025-05-23T06:35:23.129Z
65c0de12efbb14b39c97f78e
/avatars/18485f79427a35bd9e19f71b67c88dce.svg
Yuqing Yang
user
ayyyq
68301340cc0d12cd873342e2
Robin Jia
When Do LLMs Admit Their Mistakes? Understanding the Role of Model
  Belief in Retraction
Can large language models (LLMs) admit their mistakes when they should know
better? In this work, we define the behavior of acknowledging errors in
previously generated answers as "retraction" and aim to understand when and why
LLMs choose to retract. We first construct model-specific datasets to evaluate
whether a model will retract an incorrect answer that contradicts its own
parametric knowledge. While LLMs are capable of retraction, they do so only
infrequently. We demonstrate that retraction is closely tied to previously
identified indicators of models' internal belief: models fail to retract wrong
answers that they "believe" to be factually correct. Steering experiments
further demonstrate that internal belief causally influences model retraction.
In particular, when the model does not believe its answer, this not only
encourages the model to attempt to verify the answer, but also alters attention
behavior during self-verification. Finally, we demonstrate that simple
supervised fine-tuning significantly improves retraction performance by helping
the model learn more accurate internal beliefs. Code and datasets are available
on https://github.com/ayyyq/llm-retraction.
LLMs rarely retract incorrect answers they believe to be factually correct, but supervised fine-tuning can improve their retraction performance by refining their internal beliefs.
retraction model-specific datasets parametric knowledge internal belief self-verification attention behavior supervised fine-tuning