title: VideoMathQA: Benchmarking Mathematical Reasoning via Multimodal
    Understanding in Videos
summary: Mathematical reasoning in real-world video settings presents a fundamentally
  different challenge than in static images or text. It requires interpreting
  fine-grained visual information, accurately reading handwritten or digital
  text, and integrating spoken cues, often dispersed non-linearly over time. In
  such multimodal contexts, success hinges not just on perception, but on
  selectively identifying and integrating the right contextual details from a
  rich and noisy stream of content. To this end, we introduce VideoMathQA, a
  benchmark designed to evaluate whether models can perform such temporally
  extended cross-modal reasoning on videos. The benchmark spans 10 diverse
  mathematical domains, covering videos ranging from 10 seconds to over 1 hour.
  It requires models to interpret structured visual content, understand
  instructional narratives, and jointly ground concepts across visual, audio, and
  textual modalities. We employ graduate-level experts to ensure high quality,
  totaling over 920 man-hours of annotation. To reflect real-world scenarios,
  questions are designed around three core reasoning challenges: direct problem
  solving, where answers are grounded in the presented question; conceptual
  transfer, which requires applying learned methods to new problems; and deep
  instructional comprehension, involving multi-step reasoning over extended
  explanations and partially worked-out solutions. Each question includes
  multi-step reasoning annotations, enabling fine-grained diagnosis of model
  capabilities. Through this benchmark, we highlight the limitations of existing
  approaches and establish a systematic evaluation framework for models that must
  reason, rather than merely perceive, across temporally extended and
  modality-rich mathematical problem settings. Our benchmark and evaluation code
  are available at: https://mbzuai-oryx.github.io/VideoMathQA
paper_authors._id: 68424bed54a0d0e4b906baca
paper_authors.name: Hanoona Rasheed
paper_authors._id: 68424bed54a0d0e4b906bacb
paper_authors.name: Abdelrahman Shaker
paper_authors._id: 68424bed54a0d0e4b906bacc
paper_authors.name: Anqi Tang
paper_authors._id: 68424bed54a0d0e4b906bacd
paper_authors.name: Muhammad Maaz
paper_authors._id: 68424bed54a0d0e4b906bace
paper_authors.name: Ming-Hsuan Yang
paper_authors._id: 68424bed54a0d0e4b906bacf
paper_authors.name: Salman Khan
paper_authors._id: 68424bed54a0d0e4b906bad0
paper_authors.name: Fahad Khan
paper_title: VideoMathQA: Benchmarking Mathematical Reasoning via Multimodal
    Understanding in Videos
paper_summary: Mathematical reasoning in real-world video settings presents a fundamentally
  different challenge than in static images or text. It requires interpreting
  fine-grained visual information, accurately reading handwritten or digital
  text, and integrating spoken cues, often dispersed non-linearly over time. In
  such multimodal contexts, success hinges not just on perception, but on
  selectively identifying and integrating the right contextual details from a
  rich and noisy stream of content. To this end, we introduce VideoMathQA, a
  benchmark designed to evaluate whether models can perform such temporally
  extended cross-modal reasoning on videos. The benchmark spans 10 diverse
  mathematical domains, covering videos ranging from 10 seconds to over 1 hour.
  It requires models to interpret structured visual content, understand
  instructional narratives, and jointly ground concepts across visual, audio, and
  textual modalities. We employ graduate-level experts to ensure high quality,
  totaling over 920 man-hours of annotation. To reflect real-world scenarios,
  questions are designed around three core reasoning challenges: direct problem
  solving, where answers are grounded in the presented question; conceptual
  transfer, which requires applying learned methods to new problems; and deep
  instructional comprehension, involving multi-step reasoning over extended
  explanations and partially worked-out solutions. Each question includes
  multi-step reasoning annotations, enabling fine-grained diagnosis of model
  capabilities. Through this benchmark, we highlight the limitations of existing
  approaches and establish a systematic evaluation framework for models that must
  reason, rather than merely perceive, across temporally extended and
  modality-rich mathematical problem settings. Our benchmark and evaluation code
  are available at: https://mbzuai-oryx.github.io/VideoMathQA
paper_ai_summary: VideoMathQA evaluates models' ability to perform temporally extended cross-modal reasoning across various mathematical domains in video settings, addressing direct problem solving, conceptual transfer, and deep instructional comprehension.
paper_ai_keywords: VideoMathQA temporally extended cross-modal reasoning structured visual content instructional narratives modality-rich multi-step reasoning partial solutions multi-step reasoning annotations system evaluation framework