title: Virgo: A Preliminary Exploration on Reproducing o1-like MLLM
summary: Recently, slow-thinking reasoning systems, built upon large language models
  (LLMs), have garnered widespread attention by scaling the thinking time during
  inference. There is also growing interest in adapting this capability to
  multimodal large language models (MLLMs). Given that MLLMs handle more complex
  data semantics across different modalities, it is intuitively more challenging
  to implement multimodal slow-thinking systems.
    To address this issue, in this paper, we explore a straightforward approach
  by fine-tuning a capable MLLM with a small amount of textual long-form thought
  data, resulting in a multimodal slow-thinking system, Virgo (Visual reasoning
  with long thought). We find that these long-form reasoning processes, expressed
  in natural language, can be effectively transferred to MLLMs. Moreover, it
  seems that such textual reasoning data can be even more effective than visual
  reasoning data in eliciting the slow-thinking capacities of MLLMs. While this
  work is preliminary, it demonstrates that slow-thinking capacities are
  fundamentally associated with the language model component, which can be
  transferred across modalities or domains. This finding can be leveraged to
  guide the development of more powerful slow-thinking reasoning systems. We
  release our resources at https://github.com/RUCAIBox/Virgo.
paper_authors._id: 677b56b8b91035bc42259da5
paper_authors.name: Yifan Du
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T08:32:27.164Z
paper_authors._id: 61d78857a21a9b49c7e8e4a9
paper_authors.avatarUrl: /avatars/c7e7f84cad775be2d13fab8530bf21f5.svg
paper_authors.fullname: Yifan Du
paper_authors.type: user
paper_authors.user: Richard1999
paper_authors._id: 677b56b8b91035bc42259da6
paper_authors.name: Zikang Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T08:32:32.935Z
paper_authors._id: 6448dcf1b6ac93fe6512e342
paper_authors.avatarUrl: /avatars/a6441f89eabd156181bafc47c0b2f8c8.svg
paper_authors.fullname: Zikang Liu
paper_authors.type: user
paper_authors.user: JohnCage
paper_authors._id: 677b56b8b91035bc42259da7
paper_authors.name: Yifan Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T08:32:39.715Z
paper_authors._id: 65407faf81a8731a1c134c39
paper_authors.avatarUrl: /avatars/eddc718cf16d316223bcbbd2d13cf15d.svg
paper_authors.fullname: Yifan Li
paper_authors.type: user
paper_authors.user: yifanli
paper_authors._id: 677b56b8b91035bc42259da8
paper_authors.name: Wayne Xin Zhao
paper_authors._id: 677b56b8b91035bc42259da9
paper_authors.name: Yuqi Huo
paper_authors._id: 677b56b8b91035bc42259daa
paper_authors.name: Bingning Wang
paper_authors._id: 677b56b8b91035bc42259dab
paper_authors.name: Weipeng Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T08:33:31.102Z
paper_authors._id: 6501587887b370a56ad2608e
paper_authors.avatarUrl: /avatars/6779baaa8ed9032de55a2f78e1f52e20.svg
paper_authors.fullname: Wei-Peng Chen
paper_authors.type: user
paper_authors.user: whenfra
paper_authors._id: 677b56b8b91035bc42259dac
paper_authors.name: Zheng Liu
paper_authors._id: 677b56b8b91035bc42259dad
paper_authors.name: Zhongyuan Wang
paper_authors._id: 677b56b8b91035bc42259dae
paper_authors.name: Ji-Rong Wen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T08:34:39.799Z
paper_authors._id: 64b8c89052b7353d8c6a1013
paper_authors.avatarUrl: /avatars/cd59fffe81f6b07b4519540b8ff3d95f.svg
paper_authors.fullname: Ji-Rong Wen
paper_authors.type: user
paper_authors.user: jrwen
paper_title: Virgo: A Preliminary Exploration on Reproducing o1-like MLLM
paper_summary: Recently, slow-thinking reasoning systems, built upon large language models
  (LLMs), have garnered widespread attention by scaling the thinking time during
  inference. There is also growing interest in adapting this capability to
  multimodal large language models (MLLMs). Given that MLLMs handle more complex
  data semantics across different modalities, it is intuitively more challenging
  to implement multimodal slow-thinking systems.
    To address this issue, in this paper, we explore a straightforward approach
  by fine-tuning a capable MLLM with a small amount of textual long-form thought
  data, resulting in a multimodal slow-thinking system, Virgo (Visual reasoning
  with long thought). We find that these long-form reasoning processes, expressed
  in natural language, can be effectively transferred to MLLMs. Moreover, it
  seems that such textual reasoning data can be even more effective than visual
  reasoning data in eliciting the slow-thinking capacities of MLLMs. While this
  work is preliminary, it demonstrates that slow-thinking capacities are
  fundamentally associated with the language model component, which can be
  transferred across modalities or domains. This finding can be leveraged to
  guide the development of more powerful slow-thinking reasoning systems. We
  release our resources at https://github.com/RUCAIBox/Virgo.
paper_ai_summary: Fine-tuning multimodal large language models with long-form textual reasoning data enhances their slow-thinking capabilities, demonstrating the transferability of these capacities across modalities.
paper_ai_keywords: large language models multimodal large language models fine-tuning long-form thought slow-thinking reasoning visual reasoning textual reasoning natural language multimodal slow-thinking system