VideoReasonBench: Can MLLMs Perform Vision-Centric Complex Video
  Reasoning?
Recent studies have shown that long chain-of-thought (CoT) reasoning can
significantly enhance the performance of large language models (LLMs) on
complex tasks. However, this benefit is yet to be demonstrated in the domain of
video understanding, since most existing benchmarks lack the reasoning depth
required to demonstrate the advantages of extended CoT chains. While recent
efforts have proposed benchmarks aimed at video reasoning, the tasks are often
knowledge-driven and do not rely heavily on visual content. To bridge this gap,
we introduce VideoReasonBench, a benchmark designed to evaluate vision-centric,
complex video reasoning. To ensure visual richness and high reasoning
complexity, each video in VideoReasonBench depicts a sequence of fine-grained
operations on a latent state that is only visible in part of the video. The
questions evaluate three escalating levels of video reasoning skills: recalling
observed visual information, inferring the content of latent states, and
predicting information beyond the video. Under such task setting, models have
to precisely recall multiple operations in the video, and perform step-by-step
reasoning to get correct final answers for these questions. Using
VideoReasonBench, we comprehensively evaluate 18 state-of-the-art multimodal
LLMs (MLLMs), finding that most perform poorly on complex video reasoning,
e.g., GPT-4o achieves only 6.9% accuracy, while the thinking-enhanced
Gemini-2.5-Pro significantly outperforms others with 56.0% accuracy. Our
investigations on "test-time scaling" further reveal that extended thinking
budget, while offering none or minimal benefits on existing video benchmarks,
is essential for improving the performance on VideoReasonBench.
683909a29deef11aa625817c
Yuanxin Liu
683909a29deef11aa625817d
Kun Ouyang
claimed_verified
2025-05-30T06:54:22.290Z
62cd7aca7a036fc9941bb2b0
/avatars/17a4d27af0243fd7dccf06066f671461.svg
kun ouyang
user
RUBBISHLIKE
683909a29deef11aa625817e
Haoning Wu
claimed_verified
2025-06-02T07:47:29.111Z
63047ed2412a1b9d381b09c9
https://cdn-avatars.huggingface.co/v1/production/uploads/63047ed2412a1b9d381b09c9/2Ill5G0uSMyGstrawgmIb.jpeg
Haoning Wu, Teo
user
teowu
683909a29deef11aa625817f
Yi Liu
683909a29deef11aa6258180
Lin Sui
683909a29deef11aa6258181
Xinhao Li
683909a29deef11aa6258182
Yan Zhong
683909a29deef11aa6258183
Y. Charles
683909a29deef11aa6258184
Xinyu Zhou
683909a29deef11aa6258185
Xu Sun
VideoReasonBench: Can MLLMs Perform Vision-Centric Complex Video
  Reasoning?
Recent studies have shown that long chain-of-thought (CoT) reasoning can
significantly enhance the performance of large language models (LLMs) on
complex tasks. However, this benefit is yet to be demonstrated in the domain of
video understanding, since most existing benchmarks lack the reasoning depth
required to demonstrate the advantages of extended CoT chains. While recent
efforts have proposed benchmarks aimed at video reasoning, the tasks are often
knowledge-driven and do not rely heavily on visual content. To bridge this gap,
we introduce VideoReasonBench, a benchmark designed to evaluate vision-centric,
complex video reasoning. To ensure visual richness and high reasoning
complexity, each video in VideoReasonBench depicts a sequence of fine-grained
operations on a latent state that is only visible in part of the video. The
questions evaluate three escalating levels of video reasoning skills: recalling
observed visual information, inferring the content of latent states, and
predicting information beyond the video. Under such task setting, models have
to precisely recall multiple operations in the video, and perform step-by-step
reasoning to get correct final answers for these questions. Using
VideoReasonBench, we comprehensively evaluate 18 state-of-the-art multimodal
LLMs (MLLMs), finding that most perform poorly on complex video reasoning,
e.g., GPT-4o achieves only 6.9% accuracy, while the thinking-enhanced
Gemini-2.5-Pro significantly outperforms others with 56.0% accuracy. Our
investigations on "test-time scaling" further reveal that extended thinking
budget, while offering none or minimal benefits on existing video benchmarks,
is essential for improving the performance on VideoReasonBench.
A new benchmark, VideoReasonBench, evaluates complex vision-centric video reasoning, finding that extended thinking budgets are crucial for improved performance compared to existing benchmarks.
long chain-of-thought reasoning large language models video understanding VideoReasonBench vision-centric complex video reasoning latent state visual reasoning step-by-step reasoning multimodal language models thinking-enhanced models test-time scaling