title: More Thinking, Less Seeing? Assessing Amplified Hallucination in
    Multimodal Reasoning Models
summary: Test-time compute has empowered multimodal large language models to generate
  extended reasoning chains, yielding strong performance on tasks such as
  multimodal math reasoning. However, this improved reasoning ability often comes
  with increased hallucination: as generations become longer, models tend to
  drift away from image-grounded content and rely more heavily on language
  priors. Attention analysis shows that longer reasoning chains lead to reduced
  focus on visual inputs, which contributes to hallucination. To systematically
  study this phenomenon, we introduce RH-AUC, a metric that quantifies how a
  model's perception accuracy changes with reasoning length, allowing us to
  evaluate whether the model preserves visual grounding during reasoning. We also
  release RH-Bench, a diagnostic benchmark that spans a variety of multimodal
  tasks, designed to assess the trade-off between reasoning ability and
  hallucination. Our analysis reveals that (i) larger models typically achieve a
  better balance between reasoning and perception, and (ii) this balance is
  influenced more by the types and domains of training data than by its overall
  volume. These findings underscore the importance of evaluation frameworks that
  jointly consider both reasoning quality and perceptual fidelity.
paper_authors._id: 683d8d3ef63376ea8e7bf358
paper_authors.name: Chengzhi Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-02T13:39:21.988Z
paper_authors._id: 65e71f6bcd3df9b0f6b2678b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65e71f6bcd3df9b0f6b2678b/8mdex6eI80TGezot_GGI3.jpeg
paper_authors.fullname: Chengzhi Liu
paper_authors.type: user
paper_authors.user: LCZZZZ
paper_authors._id: 683d8d3ef63376ea8e7bf359
paper_authors.name: Zhongxing Xu
paper_authors._id: 683d8d3ef63376ea8e7bf35a
paper_authors.name: Qingyue Wei
paper_authors._id: 683d8d3ef63376ea8e7bf35b
paper_authors.name: Juncheng Wu
paper_authors._id: 683d8d3ef63376ea8e7bf35c
paper_authors.name: James Zou
paper_authors._id: 683d8d3ef63376ea8e7bf35d
paper_authors.name: Xin Eric Wang
paper_authors._id: 683d8d3ef63376ea8e7bf35e
paper_authors.name: Yuyin Zhou
paper_authors._id: 683d8d3ef63376ea8e7bf35f
paper_authors.name: Sheng Liu
paper_title: More Thinking, Less Seeing? Assessing Amplified Hallucination in
    Multimodal Reasoning Models
paper_summary: Test-time compute has empowered multimodal large language models to generate
  extended reasoning chains, yielding strong performance on tasks such as
  multimodal math reasoning. However, this improved reasoning ability often comes
  with increased hallucination: as generations become longer, models tend to
  drift away from image-grounded content and rely more heavily on language
  priors. Attention analysis shows that longer reasoning chains lead to reduced
  focus on visual inputs, which contributes to hallucination. To systematically
  study this phenomenon, we introduce RH-AUC, a metric that quantifies how a
  model's perception accuracy changes with reasoning length, allowing us to
  evaluate whether the model preserves visual grounding during reasoning. We also
  release RH-Bench, a diagnostic benchmark that spans a variety of multimodal
  tasks, designed to assess the trade-off between reasoning ability and
  hallucination. Our analysis reveals that (i) larger models typically achieve a
  better balance between reasoning and perception, and (ii) this balance is
  influenced more by the types and domains of training data than by its overall
  volume. These findings underscore the importance of evaluation frameworks that
  jointly consider both reasoning quality and perceptual fidelity.
paper_ai_summary: A new metric and benchmark are introduced to evaluate multimodal large language models' ability to maintain visual grounding while performing extended reasoning, revealing that larger models and specific training data types improve this balance.
paper_ai_keywords: multimodal large language models extended reasoning chains hallucination attention analysis RH-AUC perception accuracy RH-Bench multimodal tasks perceptual fidelity