title: VCRBench: Exploring Long-form Causal Reasoning Capabilities of Large
    Video Language Models
summary: Despite recent advances in video understanding, the capabilities of Large
  Video Language Models (LVLMs) to perform video-based causal reasoning remains
  underexplored, largely due to the absence of relevant and dedicated benchmarks
  for evaluating causal reasoning in visually grounded and goal-driven settings.
  To fill this gap, we introduce a novel benchmark named Video-based long-form
  Causal Reasoning (VCRBench). We create VCRBench using procedural videos of
  simple everyday activities, where the steps are deliberately shuffled with each
  clip capturing a key causal event, to test whether LVLMs can identify, reason
  about, and correctly sequence the events needed to accomplish a specific goal.
  Moreover, the benchmark is carefully designed to prevent LVLMs from exploiting
  linguistic shortcuts, as seen in multiple-choice or binary QA formats, while
  also avoiding the challenges associated with evaluating open-ended QA. Our
  evaluation of state-of-the-art LVLMs on VCRBench suggests that these models
  struggle with video-based long-form causal reasoning, primarily due to their
  difficulty in modeling long-range causal dependencies directly from visual
  observations. As a simple step toward enabling such capabilities, we propose
  Recognition-Reasoning Decomposition (RRD), a modular approach that breaks
  video-based causal reasoning into two sub-tasks of video recognition and causal
  reasoning. Our experiments on VCRBench show that RRD significantly boosts
  accuracy on VCRBench, with gains of up to 25.2%. Finally, our thorough analysis
  reveals interesting insights, for instance, that LVLMs primarily rely on
  language knowledge for complex video-based long-form causal reasoning tasks.
paper_authors._id: 6824176351679cbc704daa88
paper_authors.name: Pritam Sarkar
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-05-14T04:09:09.283Z
paper_authors._id: 6483b3d52193a1768c00c5ff
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6483b3d52193a1768c00c5ff/uZ212A-sgMr5nAEvlEgHq.jpeg
paper_authors.fullname: Pritam Sarkar
paper_authors.type: user
paper_authors.user: pritamqu
paper_authors._id: 6824176351679cbc704daa89
paper_authors.name: Ali Etemad
paper_title: VCRBench: Exploring Long-form Causal Reasoning Capabilities of Large
    Video Language Models
paper_summary: Despite recent advances in video understanding, the capabilities of Large
  Video Language Models (LVLMs) to perform video-based causal reasoning remains
  underexplored, largely due to the absence of relevant and dedicated benchmarks
  for evaluating causal reasoning in visually grounded and goal-driven settings.
  To fill this gap, we introduce a novel benchmark named Video-based long-form
  Causal Reasoning (VCRBench). We create VCRBench using procedural videos of
  simple everyday activities, where the steps are deliberately shuffled with each
  clip capturing a key causal event, to test whether LVLMs can identify, reason
  about, and correctly sequence the events needed to accomplish a specific goal.
  Moreover, the benchmark is carefully designed to prevent LVLMs from exploiting
  linguistic shortcuts, as seen in multiple-choice or binary QA formats, while
  also avoiding the challenges associated with evaluating open-ended QA. Our
  evaluation of state-of-the-art LVLMs on VCRBench suggests that these models
  struggle with video-based long-form causal reasoning, primarily due to their
  difficulty in modeling long-range causal dependencies directly from visual
  observations. As a simple step toward enabling such capabilities, we propose
  Recognition-Reasoning Decomposition (RRD), a modular approach that breaks
  video-based causal reasoning into two sub-tasks of video recognition and causal
  reasoning. Our experiments on VCRBench show that RRD significantly boosts
  accuracy on VCRBench, with gains of up to 25.2%. Finally, our thorough analysis
  reveals interesting insights, for instance, that LVLMs primarily rely on
  language knowledge for complex video-based long-form causal reasoning tasks.
paper_ai_summary: A novel benchmark called VCRBench is introduced to evaluate video-based causal reasoning in LVLMs, and a modular approach named Recognition-Reasoning Decomposition (RRD) is proposed to improve their performance.
paper_ai_keywords: Large Video Language Models LVLMs video understanding causal reasoning VCRBench procedural videos causal dependencies video recognition causal reasoning Recognition-Reasoning Decomposition RRD