title: MMR-V: What's Left Unsaid? A Benchmark for Multimodal Deep Reasoning in
    Videos
summary: The sequential structure of videos poses a challenge to the ability of
  multimodal large language models (MLLMs) to locate multi-frame evidence and
  conduct multimodal reasoning. However, existing video benchmarks mainly focus
  on understanding tasks, which only require models to match frames mentioned in
  the question (hereafter referred to as "question frame") and perceive a few
  adjacent frames. To address this gap, we propose MMR-V: A Benchmark for
  Multimodal Deep Reasoning in Videos. The benchmark is characterized by the
  following features. (1) Long-range, multi-frame reasoning: Models are required
  to infer and analyze evidence frames that may be far from the question frame.
  (2) Beyond perception: Questions cannot be answered through direct perception
  alone but require reasoning over hidden information. (3) Reliability: All tasks
  are manually annotated, referencing extensive real-world user understanding to
  align with common perceptions. (4) Confusability: Carefully designed distractor
  annotation strategies to reduce model shortcuts. MMR-V consists of 317 videos
  and 1,257 tasks. Our experiments reveal that current models still struggle with
  multi-modal reasoning; even the best-performing model, o4-mini, achieves only
  52.5% accuracy. Additionally, current reasoning enhancement strategies
  (Chain-of-Thought and scaling test-time compute) bring limited gains. Further
  analysis indicates that the CoT demanded for multi-modal reasoning differs from
  it in textual reasoning, which partly explains the limited performance gains.
  We hope that MMR-V can inspire further research into enhancing multi-modal
  reasoning capabilities.
paper_authors._id: 684106fc8cb0edba3ab212bb
paper_authors.name: Kejian Zhu
paper_authors._id: 684106fc8cb0edba3ab212bc
paper_authors.name: Zhuoran Jin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-06-05T12:49:35.145Z
paper_authors._id: 643379416c6ecd58798421b3
paper_authors.avatarUrl: /avatars/831db7eab2663abc33b176cf386b02f2.svg
paper_authors.fullname: Zhuoran Jin
paper_authors.type: user
paper_authors.user: jinzhuoran
paper_authors._id: 684106fc8cb0edba3ab212bd
paper_authors.name: Hongbang Yuan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-06-05T12:49:42.552Z
paper_authors._id: 6538d3089d66a6c304ee2f4b
paper_authors.avatarUrl: /avatars/6e93ea6c937128765e5323fb4a5d1cea.svg
paper_authors.fullname: HongbangYuan
paper_authors.type: user
paper_authors.user: HongbangYuan
paper_authors._id: 684106fc8cb0edba3ab212be
paper_authors.name: Jiachun Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-06-05T12:49:50.618Z
paper_authors._id: 64be6db0e38420aabaef8e12
paper_authors.avatarUrl: /avatars/4d71169bdfd437a8fb9a026ac0dc46de.svg
paper_authors.fullname: JIACHUN LI
paper_authors.type: user
paper_authors.user: ChristinaLJC
paper_authors._id: 684106fc8cb0edba3ab212bf
paper_authors.name: Shangqing Tu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-06-05T12:49:22.312Z
paper_authors._id: 648c48d8c0ddeee6df5b6d22
paper_authors.avatarUrl: /avatars/8706b0b16dfc332b96c91d3ced31bd0b.svg
paper_authors.fullname: Shangqing Tu
paper_authors.type: user
paper_authors.user: tsq2000
paper_authors._id: 684106fc8cb0edba3ab212c0
paper_authors.name: Pengfei Cao
paper_authors._id: 684106fc8cb0edba3ab212c1
paper_authors.name: Yubo Chen
paper_authors._id: 684106fc8cb0edba3ab212c2
paper_authors.name: Kang Liu
paper_authors._id: 684106fc8cb0edba3ab212c3
paper_authors.name: Jun Zhao
paper_title: MMR-V: What's Left Unsaid? A Benchmark for Multimodal Deep Reasoning in
    Videos
paper_summary: The sequential structure of videos poses a challenge to the ability of
  multimodal large language models (MLLMs) to locate multi-frame evidence and
  conduct multimodal reasoning. However, existing video benchmarks mainly focus
  on understanding tasks, which only require models to match frames mentioned in
  the question (hereafter referred to as "question frame") and perceive a few
  adjacent frames. To address this gap, we propose MMR-V: A Benchmark for
  Multimodal Deep Reasoning in Videos. The benchmark is characterized by the
  following features. (1) Long-range, multi-frame reasoning: Models are required
  to infer and analyze evidence frames that may be far from the question frame.
  (2) Beyond perception: Questions cannot be answered through direct perception
  alone but require reasoning over hidden information. (3) Reliability: All tasks
  are manually annotated, referencing extensive real-world user understanding to
  align with common perceptions. (4) Confusability: Carefully designed distractor
  annotation strategies to reduce model shortcuts. MMR-V consists of 317 videos
  and 1,257 tasks. Our experiments reveal that current models still struggle with
  multi-modal reasoning; even the best-performing model, o4-mini, achieves only
  52.5% accuracy. Additionally, current reasoning enhancement strategies
  (Chain-of-Thought and scaling test-time compute) bring limited gains. Further
  analysis indicates that the CoT demanded for multi-modal reasoning differs from
  it in textual reasoning, which partly explains the limited performance gains.
  We hope that MMR-V can inspire further research into enhancing multi-modal
  reasoning capabilities.
paper_ai_summary: A new benchmark, MMR-V, is proposed to challenge multimodal large language models with long-range, multi-frame reasoning and hidden information processing in videos, revealing their limitations and inspiring further research.
paper_ai_keywords: multimodal large language models MMR-V long-range multi-frame reasoning multimodal reasoning manual annotation distractor annotation strategy Chain-of-Thought