Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey
By extending the advantage of chain-of-thought (CoT) reasoning in human-like
step-by-step processes to multimodal contexts, multimodal CoT (MCoT) reasoning
has recently garnered significant research attention, especially in the
integration with multimodal large language models (MLLMs). Existing MCoT
studies design various methodologies and innovative reasoning paradigms to
address the unique challenges of image, video, speech, audio, 3D, and
structured data across different modalities, achieving extensive success in
applications such as robotics, healthcare, autonomous driving, and multimodal
generation. However, MCoT still presents distinct challenges and opportunities
that require further focus to ensure consistent thriving in this field, where,
unfortunately, an up-to-date review of this domain is lacking. To bridge this
gap, we present the first systematic survey of MCoT reasoning, elucidating the
relevant foundational concepts and definitions. We offer a comprehensive
taxonomy and an in-depth analysis of current methodologies from diverse
perspectives across various application scenarios. Furthermore, we provide
insights into existing challenges and future research directions, aiming to
foster innovation toward multimodal AGI.
67d939f6fa59a8b15aa931a8
Yaoting Wang
admin_assigned
2025-03-18T09:29:33.845Z
64ff369d9abcc85a5519b33e
/avatars/4b99cdaf5f970d930b196eddf1e5e499.svg
Yaoting Wang
user
Gh0stAR
67d939f6fa59a8b15aa931a9
Shengqiong Wu
admin_assigned
2025-03-18T09:29:41.386Z
64c139d867eff857ea51caa8
/avatars/4b7b3f41c2e2cfa21dd43bbac6e081ae.svg
Shengqiong Wu
user
ChocoWu
67d939f6fa59a8b15aa931aa
Yuecheng Zhang
claimed_verified
2025-03-18T14:58:19.876Z
6418554a0956be7233a1023e
https://cdn-avatars.huggingface.co/v1/production/uploads/6418554a0956be7233a1023e/9EKN0GoOpcDbvBDmAQEJf.png
zhang yuechen
user
julianjuaner
67d939f6fa59a8b15aa931ab
William Wang
67d939f6fa59a8b15aa931ac
Ziwei Liu
admin_assigned
2025-03-18T09:30:17.710Z
62ab1ac1d48b4d8b048a3473
https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
Ziwei Liu
user
liuziwei7
67d939f6fa59a8b15aa931ad
Jiebo Luo
67d939f6fa59a8b15aa931ae
Hao Fei
claimed_verified
2025-03-18T09:30:49.867Z
647773a1168cb428e00e9a8f
https://cdn-avatars.huggingface.co/v1/production/uploads/647773a1168cb428e00e9a8f/NiRR3ScY6Plzjibfwy1hC.jpeg
Hao Fei
user
scofield7419
Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey
By extending the advantage of chain-of-thought (CoT) reasoning in human-like
step-by-step processes to multimodal contexts, multimodal CoT (MCoT) reasoning
has recently garnered significant research attention, especially in the
integration with multimodal large language models (MLLMs). Existing MCoT
studies design various methodologies and innovative reasoning paradigms to
address the unique challenges of image, video, speech, audio, 3D, and
structured data across different modalities, achieving extensive success in
applications such as robotics, healthcare, autonomous driving, and multimodal
generation. However, MCoT still presents distinct challenges and opportunities
that require further focus to ensure consistent thriving in this field, where,
unfortunately, an up-to-date review of this domain is lacking. To bridge this
gap, we present the first systematic survey of MCoT reasoning, elucidating the
relevant foundational concepts and definitions. We offer a comprehensive
taxonomy and an in-depth analysis of current methodologies from diverse
perspectives across various application scenarios. Furthermore, we provide
insights into existing challenges and future research directions, aiming to
foster innovation toward multimodal AGI.
A systematic survey of multimodal chain-of-thought reasoning, addressing foundational concepts, methodologies, challenges, and future directions for multimodal applications.
chain-of-thought reasoning multimodal CoT multimodal large language models image video speech audio 3D structured data robotics healthcare autonomous driving multimodal generation multimodal AGI