VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal
  Large Language Models
Visual reasoning is a core component of human intelligence and a critical
capability for advanced multimodal models. Yet current reasoning evaluations of
multimodal large language models (MLLMs) often rely on text descriptions and
allow language-based reasoning shortcuts, failing to measure genuine
vision-centric reasoning. To address this, we introduce VisuLogic: a benchmark
of 1,000 human-verified problems across six categories (e.g., quantitative
shifts, spatial relations, attribute comparisons). These various types of
questions can be evaluated to assess the visual reasoning capabilities of MLLMs
from multiple perspectives. We evaluate leading MLLMs on this benchmark and
analyze their results to identify common failure modes. Most models score below
30% accuracy-only slightly above the 25% random baseline and far below the
51.4% achieved by humans-revealing significant gaps in visual reasoning.
Furthermore, we provide a supplementary training dataset and a
reinforcement-learning baseline to support further progress.
68070d3b5035e6d88636ae13
Weiye Xu
claimed_verified
2025-04-24T09:11:19.332Z
649e5ee29420f68cf1c1470e
/avatars/7f6d1ec4fb3f85351e88044016d8ab42.svg
Xu Wayen
user
wilye
68070d3b5035e6d88636ae14
Jiahao Wang
claimed_verified
2025-04-24T09:11:17.358Z
664b4a748dd1bfb5a3a970fe
/avatars/37aa9332ab3e8fbb6ae30b875a7e0e5a.svg
Jiahao Wang
user
GenuineWWD
68070d3b5035e6d88636ae15
Weiyun Wang
admin_assigned
2025-04-24T11:20:27.920Z
619507e7b74b6c591f794340
https://cdn-avatars.huggingface.co/v1/production/uploads/619507e7b74b6c591f794340/JbPDoy6Ko1V1-6oJJwFV8.jpeg
Weiyun Wang
user
Weiyun1025
68070d3b5035e6d88636ae16
Zhe Chen
68070d3b5035e6d88636ae17
Wengang Zhou
68070d3b5035e6d88636ae18
Aijun Yang
68070d3b5035e6d88636ae19
Lewei Lu
admin_assigned
2025-04-24T11:20:51.831Z
65ead3ea908526a39082e641
/avatars/dcf870695fd56b06ca03d82f831e9019.svg
Lewei Lu
user
luotto
68070d3b5035e6d88636ae1a
Houqiang Li
68070d3b5035e6d88636ae1b
Xiaohua Wang
68070d3b5035e6d88636ae1c
Xizhou Zhu
admin_assigned
2025-04-24T11:21:11.459Z
64ae2359179421d320b1694b
/avatars/c387a75191005bcaa473091de5383a10.svg
Xizhou Zhu
user
Einsiedler
68070d3b5035e6d88636ae1d
Wenhai Wang
admin_assigned
2025-04-24T11:21:24.337Z
64d1c560c0c627dfa71bdbe0
/avatars/f42794fe25bffcd870a1bcee69b95298.svg
wenhai.wang
user
wangwhcore
68070d3b5035e6d88636ae1e
Jifeng Dai
admin_assigned
2025-04-24T11:21:31.315Z
64686f7172d9180d4ac8b4e4
/avatars/db67dd6c4b2b41054ddcce5a18ade6f8.svg
Jifeng Dai
user
daijifeng
68070d3b5035e6d88636ae1f
Jinguo Zhu
VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal
  Large Language Models
Visual reasoning is a core component of human intelligence and a critical
capability for advanced multimodal models. Yet current reasoning evaluations of
multimodal large language models (MLLMs) often rely on text descriptions and
allow language-based reasoning shortcuts, failing to measure genuine
vision-centric reasoning. To address this, we introduce VisuLogic: a benchmark
of 1,000 human-verified problems across six categories (e.g., quantitative
shifts, spatial relations, attribute comparisons). These various types of
questions can be evaluated to assess the visual reasoning capabilities of MLLMs
from multiple perspectives. We evaluate leading MLLMs on this benchmark and
analyze their results to identify common failure modes. Most models score below
30% accuracy-only slightly above the 25% random baseline and far below the
51.4% achieved by humans-revealing significant gaps in visual reasoning.
Furthermore, we provide a supplementary training dataset and a
reinforcement-learning baseline to support further progress.
VisuLogic is a benchmark to evaluate genuine vision-centric reasoning in multimodal large language models, revealing significant performance gaps compared to human accuracy.
multimodal large language models visual reasoning human-verified problems spatial relations quantitative shifts attribute comparisons reinforcement-learning baseline