Perception, Reason, Think, and Plan: A Survey on Large Multimodal
  Reasoning Models
Reasoning lies at the heart of intelligence, shaping the ability to make
decisions, draw conclusions, and generalize across domains. In artificial
intelligence, as systems increasingly operate in open, uncertain, and
multimodal environments, reasoning becomes essential for enabling robust and
adaptive behavior. Large Multimodal Reasoning Models (LMRMs) have emerged as a
promising paradigm, integrating modalities such as text, images, audio, and
video to support complex reasoning capabilities and aiming to achieve
comprehensive perception, precise understanding, and deep reasoning. As
research advances, multimodal reasoning has rapidly evolved from modular,
perception-driven pipelines to unified, language-centric frameworks that offer
more coherent cross-modal understanding. While instruction tuning and
reinforcement learning have improved model reasoning, significant challenges
remain in omni-modal generalization, reasoning depth, and agentic behavior. To
address these issues, we present a comprehensive and structured survey of
multimodal reasoning research, organized around a four-stage developmental
roadmap that reflects the field's shifting design philosophies and emerging
capabilities. First, we review early efforts based on task-specific modules,
where reasoning was implicitly embedded across stages of representation,
alignment, and fusion. Next, we examine recent approaches that unify reasoning
into multimodal LLMs, with advances such as Multimodal Chain-of-Thought (MCoT)
and multimodal reinforcement learning enabling richer and more structured
reasoning chains. Finally, drawing on empirical insights from challenging
benchmarks and experimental cases of OpenAI O3 and O4-mini, we discuss the
conceptual direction of native large multimodal reasoning models (N-LMRMs),
which aim to support scalable, agentic, and adaptive reasoning and planning in
complex, real-world environments.
681dbb9988ca86d430f1d0d2
Yunxin Li
admin_assigned
2025-05-09T09:05:42.761Z
62fdb01bc1588e1d4c6c1a7c
/avatars/bd03085995b1c34e0ac8a845cf2c4e83.svg
Yunxin Li
user
YunxinLi
681dbb9988ca86d430f1d0d3
Zhenyu Liu
claimed_verified
2025-05-09T10:08:42.493Z
64380ae1819f3ab20d17431b
/avatars/a36b073c1c783102ddb455204fd816bd.svg
ZhenyuLiu
user
foggyforest
681dbb9988ca86d430f1d0d4
Zitao Li
claimed_verified
2025-05-09T08:33:26.702Z
67ecc6a08647cfa1775a9fda
/avatars/bb15abd7a3d2c51380b0b1f819ef76e2.svg
Zitao Li
user
TerenceL-TL
681dbb9988ca86d430f1d0d5
Xuanyu Zhang
claimed_verified
2025-05-09T10:36:03.450Z
676fcc8b98612a870ddb848a
/avatars/7da1b77da1d18758b08d49436c5971c1.svg
xuanyu
user
xyidealist
681dbb9988ca86d430f1d0d6
Zhenran Xu
claimed_verified
2025-05-09T10:08:40.339Z
639c379cdb7c5f35004066cb
/avatars/3e435506ee85aa7d2d0ec2174a07462f.svg
Zhenran Xu
user
imryanxu
681dbb9988ca86d430f1d0d7
Xinyu Chen
claimed_verified
2025-05-09T13:54:43.127Z
64d9da538767727dff1e8f19
/avatars/aaad38795007c6cbcb94c7eed1706e51.svg
Yu
user
Ghaser
681dbb9988ca86d430f1d0d8
Haoyuan Shi
claimed_verified
2025-06-13T07:42:30.141Z
652fb8bcc9dd2692a25ef2e3
/avatars/461e6cc1c3441cde18192b080b0b8576.svg
Haoyuan Shi
user
MrSunshy
681dbb9988ca86d430f1d0d9
Shenyuan Jiang
681dbb9988ca86d430f1d0da
Xintong Wang
claimed_verified
2025-05-12T06:50:56.441Z
674879885d97f2d66a14006a
/avatars/cdeff44a898560cb460223f30922b081.svg
Xintong Wang
user
shanewang
681dbb9988ca86d430f1d0db
Jifang Wang
681dbb9988ca86d430f1d0dc
Shouzheng Huang
681dbb9988ca86d430f1d0dd
Xinping Zhao
681dbb9988ca86d430f1d0de
Borui Jiang
681dbb9988ca86d430f1d0df
Lanqing Hong
681dbb9988ca86d430f1d0e0
Longyue Wang
681dbb9988ca86d430f1d0e1
Zhuotao Tian
681dbb9988ca86d430f1d0e2
Baoxing Huai
681dbb9988ca86d430f1d0e3
Wenhan Luo
681dbb9988ca86d430f1d0e4
Weihua Luo
681dbb9988ca86d430f1d0e5
Zheng Zhang
681dbb9988ca86d430f1d0e6
Baotian Hu
681dbb9988ca86d430f1d0e7
Min Zhang
Perception, Reason, Think, and Plan: A Survey on Large Multimodal
  Reasoning Models
Reasoning lies at the heart of intelligence, shaping the ability to make
decisions, draw conclusions, and generalize across domains. In artificial
intelligence, as systems increasingly operate in open, uncertain, and
multimodal environments, reasoning becomes essential for enabling robust and
adaptive behavior. Large Multimodal Reasoning Models (LMRMs) have emerged as a
promising paradigm, integrating modalities such as text, images, audio, and
video to support complex reasoning capabilities and aiming to achieve
comprehensive perception, precise understanding, and deep reasoning. As
research advances, multimodal reasoning has rapidly evolved from modular,
perception-driven pipelines to unified, language-centric frameworks that offer
more coherent cross-modal understanding. While instruction tuning and
reinforcement learning have improved model reasoning, significant challenges
remain in omni-modal generalization, reasoning depth, and agentic behavior. To
address these issues, we present a comprehensive and structured survey of
multimodal reasoning research, organized around a four-stage developmental
roadmap that reflects the field's shifting design philosophies and emerging
capabilities. First, we review early efforts based on task-specific modules,
where reasoning was implicitly embedded across stages of representation,
alignment, and fusion. Next, we examine recent approaches that unify reasoning
into multimodal LLMs, with advances such as Multimodal Chain-of-Thought (MCoT)
and multimodal reinforcement learning enabling richer and more structured
reasoning chains. Finally, drawing on empirical insights from challenging
benchmarks and experimental cases of OpenAI O3 and O4-mini, we discuss the
conceptual direction of native large multimodal reasoning models (N-LMRMs),
which aim to support scalable, agentic, and adaptive reasoning and planning in
complex, real-world environments.
This survey outlines the development of multimodal reasoning models, progressing from task-specific modules to unified frameworks, and discusses emerging capabilities and challenges in integrating reasoning across diverse data types.
Multimodal Reasoning Models LMRMs multimodal LLMs Multimodal Chain-of-Thought multimodal reinforcement learning OpenAI O3 OpenAI O4-mini native large multimodal reasoning models N-LMRMs