title: Visionary-R1: Mitigating Shortcuts in Visual Reasoning with
    Reinforcement Learning
summary: Learning general-purpose reasoning capabilities has long been a challenging
  problem in AI. Recent research in large language models (LLMs), such as
  DeepSeek-R1, has shown that reinforcement learning techniques like GRPO can
  enable pre-trained LLMs to develop reasoning capabilities using simple
  question-answer pairs. In this paper, we aim to train visual language models
  (VLMs) to perform reasoning on image data through reinforcement learning and
  visual question-answer pairs, without any explicit chain-of-thought (CoT)
  supervision. Our findings indicate that simply applying reinforcement learning
  to a VLM -- by prompting the model to produce a reasoning chain before
  providing an answer -- can lead the model to develop shortcuts from easy
  questions, thereby reducing its ability to generalize across unseen data
  distributions. We argue that the key to mitigating shortcut learning is to
  encourage the model to interpret images prior to reasoning. Therefore, we train
  the model to adhere to a caption-reason-answer output format: initially
  generating a detailed caption for an image, followed by constructing an
  extensive reasoning chain. When trained on 273K CoT-free visual question-answer
  pairs and using only reinforcement learning, our model, named Visionary-R1,
  outperforms strong multimodal models, such as GPT-4o, Claude3.5-Sonnet, and
  Gemini-1.5-Pro, on multiple visual reasoning benchmarks.
paper_authors._id: 682d718306291bf11fcf69a3
paper_authors.name: Jiaer Xia
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:14:20.493Z
paper_authors._id: 664da76e4eb4c91c8c32cc06
paper_authors.avatarUrl: /avatars/050bb6e4136f8a1645fef277ad08c7fc.svg
paper_authors.fullname: Jiaer Xia
paper_authors.type: user
paper_authors.user: Jiaer-Xia
paper_authors._id: 682d718306291bf11fcf69a4
paper_authors.name: Yuhang Zang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-21T08:39:56.735Z
paper_authors._id: 63859cf3b2906edaf83af9f0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63859cf3b2906edaf83af9f0/mCgynEtoXdILvzyoPexii.png
paper_authors.fullname: Yuhang Zang
paper_authors.type: user
paper_authors.user: yuhangzang
paper_authors._id: 682d718306291bf11fcf69a5
paper_authors.name: Peng Gao
paper_authors._id: 682d718306291bf11fcf69a6
paper_authors.name: Yixuan Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-03T08:51:33.410Z
paper_authors._id: 654c865e17d83697c75f3f90
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/654c865e17d83697c75f3f90/JW79nXWaCKmw2HmELshEw.png
paper_authors.fullname: JiaerXia
paper_authors.type: user
paper_authors.user: JiaerX
paper_authors._id: 682d718306291bf11fcf69a7
paper_authors.name: Kaiyang Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:14:28.433Z
paper_authors._id: 62ac6656de8bfbb93094b8fd
paper_authors.avatarUrl: /avatars/1d8f445b6d5f9d43ddab81056bcb141e.svg
paper_authors.fullname: Kaiyang Zhou
paper_authors.type: user
paper_authors.user: kaiyangzhou
paper_title: Visionary-R1: Mitigating Shortcuts in Visual Reasoning with
    Reinforcement Learning
paper_summary: Learning general-purpose reasoning capabilities has long been a challenging
  problem in AI. Recent research in large language models (LLMs), such as
  DeepSeek-R1, has shown that reinforcement learning techniques like GRPO can
  enable pre-trained LLMs to develop reasoning capabilities using simple
  question-answer pairs. In this paper, we aim to train visual language models
  (VLMs) to perform reasoning on image data through reinforcement learning and
  visual question-answer pairs, without any explicit chain-of-thought (CoT)
  supervision. Our findings indicate that simply applying reinforcement learning
  to a VLM -- by prompting the model to produce a reasoning chain before
  providing an answer -- can lead the model to develop shortcuts from easy
  questions, thereby reducing its ability to generalize across unseen data
  distributions. We argue that the key to mitigating shortcut learning is to
  encourage the model to interpret images prior to reasoning. Therefore, we train
  the model to adhere to a caption-reason-answer output format: initially
  generating a detailed caption for an image, followed by constructing an
  extensive reasoning chain. When trained on 273K CoT-free visual question-answer
  pairs and using only reinforcement learning, our model, named Visionary-R1,
  outperforms strong multimodal models, such as GPT-4o, Claude3.5-Sonnet, and
  Gemini-1.5-Pro, on multiple visual reasoning benchmarks.
paper_ai_summary: Reinforcement learning applied to visual language models with image captions and reasoning chains leads to improved performance on visual reasoning benchmarks compared to multimodal models.
paper_ai_keywords: reinforcement learning visual language models visual question-answer pairs chain-of-thought caption-reason-answer visual reasoning benchmarks