MINT-CoT: Enabling Interleaved Visual Tokens in Mathematical
  Chain-of-Thought Reasoning
Chain-of-Thought (CoT) has widely enhanced mathematical reasoning in Large
Language Models (LLMs), but it still remains challenging for extending it to
multimodal domains. Existing works either adopt a similar textual reasoning for
image input, or seek to interleave visual signals into mathematical CoT.
However, they face three key limitations for math problem-solving: reliance on
coarse-grained box-shaped image regions, limited perception of vision encoders
on math content, and dependence on external capabilities for visual
modification. In this paper, we propose MINT-CoT, introducing Mathematical
INterleaved Tokens for Chain-of-Thought visual reasoning. MINT-CoT adaptively
interleaves relevant visual tokens into textual reasoning steps via an
Interleave Token, which dynamically selects visual regions of any shapes within
math figures. To empower this capability, we construct the MINT-CoT dataset,
containing 54K mathematical problems aligning each reasoning step with visual
regions at the token level, accompanied by a rigorous data generation pipeline.
We further present a three-stage MINT-CoT training strategy, progressively
combining text-only CoT SFT, interleaved CoT SFT, and interleaved CoT RL, which
derives our MINT-CoT-7B model. Extensive experiments demonstrate the
effectiveness of our method for effective visual interleaved reasoning in
mathematical domains, where MINT-CoT-7B outperforms the baseline model by
+34.08% on MathVista, +28.78% on GeoQA, and +23.2% on MMStar, respectively. Our
code and data are available at https://github.com/xinyan-cxy/MINT-CoT
684260765bfed1b94a9cc307
Xinyan Chen
claimed_verified
2025-06-06T07:40:42.546Z
647c7a4ed412b3b376572a00
/avatars/9cc310fd3f9e3f211475816ed9b0cdaa.svg
Xinyan Chen
user
xy06
684260765bfed1b94a9cc308
Renrui Zhang
684260765bfed1b94a9cc309
Dongzhi Jiang
claimed_verified
2025-06-06T07:40:40.308Z
6349214f8146350b3a4c5cdf
/avatars/cfd24caac9a87efb528d0f4c375932bc.svg
Dongzhi Jiang
user
CaraJ
684260765bfed1b94a9cc30a
Aojun Zhou
684260765bfed1b94a9cc30b
Shilin Yan
684260765bfed1b94a9cc30c
Weifeng Lin
684260765bfed1b94a9cc30d
Hongsheng Li
MINT-CoT: Enabling Interleaved Visual Tokens in Mathematical
  Chain-of-Thought Reasoning
Chain-of-Thought (CoT) has widely enhanced mathematical reasoning in Large
Language Models (LLMs), but it still remains challenging for extending it to
multimodal domains. Existing works either adopt a similar textual reasoning for
image input, or seek to interleave visual signals into mathematical CoT.
However, they face three key limitations for math problem-solving: reliance on
coarse-grained box-shaped image regions, limited perception of vision encoders
on math content, and dependence on external capabilities for visual
modification. In this paper, we propose MINT-CoT, introducing Mathematical
INterleaved Tokens for Chain-of-Thought visual reasoning. MINT-CoT adaptively
interleaves relevant visual tokens into textual reasoning steps via an
Interleave Token, which dynamically selects visual regions of any shapes within
math figures. To empower this capability, we construct the MINT-CoT dataset,
containing 54K mathematical problems aligning each reasoning step with visual
regions at the token level, accompanied by a rigorous data generation pipeline.
We further present a three-stage MINT-CoT training strategy, progressively
combining text-only CoT SFT, interleaved CoT SFT, and interleaved CoT RL, which
derives our MINT-CoT-7B model. Extensive experiments demonstrate the
effectiveness of our method for effective visual interleaved reasoning in
mathematical domains, where MINT-CoT-7B outperforms the baseline model by
+34.08% on MathVista, +28.78% on GeoQA, and +23.2% on MMStar, respectively. Our
code and data are available at https://github.com/xinyan-cxy/MINT-CoT
MINT-CoT enhances multimodal mathematical reasoning by interleaving visual tokens into textual chain-of-thought steps, enabling flexible visual perception and improved problem-solving.
Chain-of-Thought Large Language Models multimodal domains textual reasoning visual signals image input vision encoders math content visual modification Mathematical INterleaved Tokens Interleave Token visual regions token level MINT-CoT dataset text-only CoT SFT interleaved CoT SFT interleaved CoT RL MINT-CoT-7B MathVista GeoQA MMStar