title: Mind with Eyes: from Language Reasoning to Multimodal Reasoning
summary: Language models have recently advanced into the realm of reasoning, yet it is
  through multimodal reasoning that we can fully unlock the potential to achieve
  more comprehensive, human-like cognitive capabilities. This survey provides a
  systematic overview of the recent multimodal reasoning approaches, categorizing
  them into two levels: language-centric multimodal reasoning and collaborative
  multimodal reasoning. The former encompasses one-pass visual perception and
  active visual perception, where vision primarily serves a supporting role in
  language reasoning. The latter involves action generation and state update
  within reasoning process, enabling a more dynamic interaction between
  modalities. Furthermore, we analyze the technical evolution of these methods,
  discuss their inherent challenges, and introduce key benchmark tasks and
  evaluation metrics for assessing multimodal reasoning performance. Finally, we
  provide insights into future research directions from the following two
  perspectives: (i) from visual-language reasoning to omnimodal reasoning and
  (ii) from multimodal reasoning to multimodal agents. This survey aims to
  provide a structured overview that will inspire further advancements in
  multimodal reasoning research.
paper_authors._id: 67e249babf8cc1d141a54299
paper_authors.name: Zhiyu Lin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-25T08:19:25.946Z
paper_authors._id: 6489b2b9a9d27ef5f8c824c7
paper_authors.avatarUrl: /avatars/14655bb803ba0db259b7a52ea96fcd09.svg
paper_authors.fullname: zyllin
paper_authors.type: user
paper_authors.user: zygg
paper_authors._id: 67e249babf8cc1d141a5429a
paper_authors.name: Yifei Gao
paper_authors._id: 67e249babf8cc1d141a5429b
paper_authors.name: Xian Zhao
paper_authors._id: 67e249babf8cc1d141a5429c
paper_authors.name: Yunfan Yang
paper_authors._id: 67e249babf8cc1d141a5429d
paper_authors.name: Jitao Sang
paper_title: Mind with Eyes: from Language Reasoning to Multimodal Reasoning
paper_summary: Language models have recently advanced into the realm of reasoning, yet it is
  through multimodal reasoning that we can fully unlock the potential to achieve
  more comprehensive, human-like cognitive capabilities. This survey provides a
  systematic overview of the recent multimodal reasoning approaches, categorizing
  them into two levels: language-centric multimodal reasoning and collaborative
  multimodal reasoning. The former encompasses one-pass visual perception and
  active visual perception, where vision primarily serves a supporting role in
  language reasoning. The latter involves action generation and state update
  within reasoning process, enabling a more dynamic interaction between
  modalities. Furthermore, we analyze the technical evolution of these methods,
  discuss their inherent challenges, and introduce key benchmark tasks and
  evaluation metrics for assessing multimodal reasoning performance. Finally, we
  provide insights into future research directions from the following two
  perspectives: (i) from visual-language reasoning to omnimodal reasoning and
  (ii) from multimodal reasoning to multimodal agents. This survey aims to
  provide a structured overview that will inspire further advancements in
  multimodal reasoning research.
paper_ai_summary: This survey categorizes and analyzes multimodal reasoning approaches, highlighting challenges and future directions in visual-language reasoning and multimodal agents.
paper_ai_keywords: multimodal reasoning language-centric multimodal reasoning collaborative multimodal reasoning one-pass visual perception active visual perception action generation state update benchmark tasks evaluation metrics omnimodal reasoning multimodal agents