Mind with Eyes: from Language Reasoning to Multimodal Reasoning
Language models have recently advanced into the realm of reasoning, yet it is
through multimodal reasoning that we can fully unlock the potential to achieve
more comprehensive, human-like cognitive capabilities. This survey provides a
systematic overview of the recent multimodal reasoning approaches, categorizing
them into two levels: language-centric multimodal reasoning and collaborative
multimodal reasoning. The former encompasses one-pass visual perception and
active visual perception, where vision primarily serves a supporting role in
language reasoning. The latter involves action generation and state update
within reasoning process, enabling a more dynamic interaction between
modalities. Furthermore, we analyze the technical evolution of these methods,
discuss their inherent challenges, and introduce key benchmark tasks and
evaluation metrics for assessing multimodal reasoning performance. Finally, we
provide insights into future research directions from the following two
perspectives: (i) from visual-language reasoning to omnimodal reasoning and
(ii) from multimodal reasoning to multimodal agents. This survey aims to
provide a structured overview that will inspire further advancements in
multimodal reasoning research.
67e249babf8cc1d141a54299
Zhiyu Lin
claimed_verified
2025-03-25T08:19:25.946Z
6489b2b9a9d27ef5f8c824c7
/avatars/14655bb803ba0db259b7a52ea96fcd09.svg
zyllin
user
zygg
67e249babf8cc1d141a5429a
Yifei Gao
67e249babf8cc1d141a5429b
Xian Zhao
67e249babf8cc1d141a5429c
Yunfan Yang
67e249babf8cc1d141a5429d
Jitao Sang
Mind with Eyes: from Language Reasoning to Multimodal Reasoning
Language models have recently advanced into the realm of reasoning, yet it is
through multimodal reasoning that we can fully unlock the potential to achieve
more comprehensive, human-like cognitive capabilities. This survey provides a
systematic overview of the recent multimodal reasoning approaches, categorizing
them into two levels: language-centric multimodal reasoning and collaborative
multimodal reasoning. The former encompasses one-pass visual perception and
active visual perception, where vision primarily serves a supporting role in
language reasoning. The latter involves action generation and state update
within reasoning process, enabling a more dynamic interaction between
modalities. Furthermore, we analyze the technical evolution of these methods,
discuss their inherent challenges, and introduce key benchmark tasks and
evaluation metrics for assessing multimodal reasoning performance. Finally, we
provide insights into future research directions from the following two
perspectives: (i) from visual-language reasoning to omnimodal reasoning and
(ii) from multimodal reasoning to multimodal agents. This survey aims to
provide a structured overview that will inspire further advancements in
multimodal reasoning research.
This survey categorizes and analyzes multimodal reasoning approaches, highlighting challenges and future directions in visual-language reasoning and multimodal agents.
multimodal reasoning language-centric multimodal reasoning collaborative multimodal reasoning one-pass visual perception active visual perception action generation state update benchmark tasks evaluation metrics omnimodal reasoning multimodal agents