title: Are Reasoning Models More Prone to Hallucination?
summary: Recently evolved large reasoning models (LRMs) show powerful performance in
  solving complex tasks with long chain-of-thought (CoT) reasoning capability. As
  these LRMs are mostly developed by post-training on formal reasoning tasks,
  whether they generalize the reasoning capability to help reduce hallucination
  in fact-seeking tasks remains unclear and debated. For instance, DeepSeek-R1
  reports increased performance on SimpleQA, a fact-seeking benchmark, while
  OpenAI-o3 observes even severer hallucination. This discrepancy naturally
  raises the following research question: Are reasoning models more prone to
  hallucination? This paper addresses the question from three perspectives. (1)
  We first conduct a holistic evaluation for the hallucination in LRMs. Our
  analysis reveals that LRMs undergo a full post-training pipeline with cold
  start supervised fine-tuning (SFT) and verifiable reward RL generally alleviate
  their hallucination. In contrast, both distillation alone and RL training
  without cold start fine-tuning introduce more nuanced hallucinations. (2) To
  explore why different post-training pipelines alters the impact on
  hallucination in LRMs, we conduct behavior analysis. We characterize two
  critical cognitive behaviors that directly affect the factuality of a LRM: Flaw
  Repetition, where the surface-level reasoning attempts repeatedly follow the
  same underlying flawed logic, and Think-Answer Mismatch, where the final answer
  fails to faithfully match the previous CoT process. (3) Further, we investigate
  the mechanism behind the hallucination of LRMs from the perspective of model
  uncertainty. We find that increased hallucination of LRMs is usually associated
  with the misalignment between model uncertainty and factual accuracy. Our work
  provides an initial understanding of the hallucination in LRMs.
paper_authors._id: 683946c845636acda08ed401
paper_authors.name: Zijun Yao
paper_authors._id: 683946c845636acda08ed402
paper_authors.name: Yantao Liu
paper_authors._id: 683946c845636acda08ed403
paper_authors.name: Yanxu Chen
paper_authors._id: 683946c845636acda08ed404
paper_authors.name: Jianhui Chen
paper_authors._id: 683946c845636acda08ed405
paper_authors.name: Junfeng Fang
paper_authors._id: 683946c845636acda08ed406
paper_authors.name: Lei Hou
paper_authors._id: 683946c845636acda08ed407
paper_authors.name: Juanzi Li
paper_authors._id: 683946c845636acda08ed408
paper_authors.name: Tat-Seng Chua
paper_title: Are Reasoning Models More Prone to Hallucination?
paper_summary: Recently evolved large reasoning models (LRMs) show powerful performance in
  solving complex tasks with long chain-of-thought (CoT) reasoning capability. As
  these LRMs are mostly developed by post-training on formal reasoning tasks,
  whether they generalize the reasoning capability to help reduce hallucination
  in fact-seeking tasks remains unclear and debated. For instance, DeepSeek-R1
  reports increased performance on SimpleQA, a fact-seeking benchmark, while
  OpenAI-o3 observes even severer hallucination. This discrepancy naturally
  raises the following research question: Are reasoning models more prone to
  hallucination? This paper addresses the question from three perspectives. (1)
  We first conduct a holistic evaluation for the hallucination in LRMs. Our
  analysis reveals that LRMs undergo a full post-training pipeline with cold
  start supervised fine-tuning (SFT) and verifiable reward RL generally alleviate
  their hallucination. In contrast, both distillation alone and RL training
  without cold start fine-tuning introduce more nuanced hallucinations. (2) To
  explore why different post-training pipelines alters the impact on
  hallucination in LRMs, we conduct behavior analysis. We characterize two
  critical cognitive behaviors that directly affect the factuality of a LRM: Flaw
  Repetition, where the surface-level reasoning attempts repeatedly follow the
  same underlying flawed logic, and Think-Answer Mismatch, where the final answer
  fails to faithfully match the previous CoT process. (3) Further, we investigate
  the mechanism behind the hallucination of LRMs from the perspective of model
  uncertainty. We find that increased hallucination of LRMs is usually associated
  with the misalignment between model uncertainty and factual accuracy. Our work
  provides an initial understanding of the hallucination in LRMs.
paper_ai_summary: Large reasoning models exhibit varying susceptibility to hallucination depending on post-training pipelines, revealing critical cognitive behaviors and uncertainty misalignment as contributing factors.
paper_ai_keywords: large reasoning models chain-of-thought post-training formal reasoning tasks fact-seeking tasks DeepSeek-R1 OpenAI-o3 hallucination cold start supervised fine-tuning verifiable reward RL distillation Flaw Repetition Think-Answer Mismatch model uncertainty factual accuracy