title: VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal
    Large Language Models
summary: Visual reasoning is a core component of human intelligence and a critical
  capability for advanced multimodal models. Yet current reasoning evaluations of
  multimodal large language models (MLLMs) often rely on text descriptions and
  allow language-based reasoning shortcuts, failing to measure genuine
  vision-centric reasoning. To address this, we introduce VisuLogic: a benchmark
  of 1,000 human-verified problems across six categories (e.g., quantitative
  shifts, spatial relations, attribute comparisons). These various types of
  questions can be evaluated to assess the visual reasoning capabilities of MLLMs
  from multiple perspectives. We evaluate leading MLLMs on this benchmark and
  analyze their results to identify common failure modes. Most models score below
  30% accuracy-only slightly above the 25% random baseline and far below the
  51.4% achieved by humans-revealing significant gaps in visual reasoning.
  Furthermore, we provide a supplementary training dataset and a
  reinforcement-learning baseline to support further progress.
paper_authors._id: 68070d3b5035e6d88636ae13
paper_authors.name: Weiye Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-24T09:11:19.332Z
paper_authors._id: 649e5ee29420f68cf1c1470e
paper_authors.avatarUrl: /avatars/7f6d1ec4fb3f85351e88044016d8ab42.svg
paper_authors.fullname: Xu Wayen
paper_authors.type: user
paper_authors.user: wilye
paper_authors._id: 68070d3b5035e6d88636ae14
paper_authors.name: Jiahao Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-24T09:11:17.358Z
paper_authors._id: 664b4a748dd1bfb5a3a970fe
paper_authors.avatarUrl: /avatars/37aa9332ab3e8fbb6ae30b875a7e0e5a.svg
paper_authors.fullname: Jiahao Wang
paper_authors.type: user
paper_authors.user: GenuineWWD
paper_authors._id: 68070d3b5035e6d88636ae15
paper_authors.name: Weiyun Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-24T11:20:27.920Z
paper_authors._id: 619507e7b74b6c591f794340
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/619507e7b74b6c591f794340/JbPDoy6Ko1V1-6oJJwFV8.jpeg
paper_authors.fullname: Weiyun Wang
paper_authors.type: user
paper_authors.user: Weiyun1025
paper_authors._id: 68070d3b5035e6d88636ae16
paper_authors.name: Zhe Chen
paper_authors._id: 68070d3b5035e6d88636ae17
paper_authors.name: Wengang Zhou
paper_authors._id: 68070d3b5035e6d88636ae18
paper_authors.name: Aijun Yang
paper_authors._id: 68070d3b5035e6d88636ae19
paper_authors.name: Lewei Lu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-24T11:20:51.831Z
paper_authors._id: 65ead3ea908526a39082e641
paper_authors.avatarUrl: /avatars/dcf870695fd56b06ca03d82f831e9019.svg
paper_authors.fullname: Lewei Lu
paper_authors.type: user
paper_authors.user: luotto
paper_authors._id: 68070d3b5035e6d88636ae1a
paper_authors.name: Houqiang Li
paper_authors._id: 68070d3b5035e6d88636ae1b
paper_authors.name: Xiaohua Wang
paper_authors._id: 68070d3b5035e6d88636ae1c
paper_authors.name: Xizhou Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-24T11:21:11.459Z
paper_authors._id: 64ae2359179421d320b1694b
paper_authors.avatarUrl: /avatars/c387a75191005bcaa473091de5383a10.svg
paper_authors.fullname: Xizhou Zhu
paper_authors.type: user
paper_authors.user: Einsiedler
paper_authors._id: 68070d3b5035e6d88636ae1d
paper_authors.name: Wenhai Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-24T11:21:24.337Z
paper_authors._id: 64d1c560c0c627dfa71bdbe0
paper_authors.avatarUrl: /avatars/f42794fe25bffcd870a1bcee69b95298.svg
paper_authors.fullname: wenhai.wang
paper_authors.type: user
paper_authors.user: wangwhcore
paper_authors._id: 68070d3b5035e6d88636ae1e
paper_authors.name: Jifeng Dai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-24T11:21:31.315Z
paper_authors._id: 64686f7172d9180d4ac8b4e4
paper_authors.avatarUrl: /avatars/db67dd6c4b2b41054ddcce5a18ade6f8.svg
paper_authors.fullname: Jifeng Dai
paper_authors.type: user
paper_authors.user: daijifeng
paper_authors._id: 68070d3b5035e6d88636ae1f
paper_authors.name: Jinguo Zhu
paper_title: VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal
    Large Language Models
paper_summary: Visual reasoning is a core component of human intelligence and a critical
  capability for advanced multimodal models. Yet current reasoning evaluations of
  multimodal large language models (MLLMs) often rely on text descriptions and
  allow language-based reasoning shortcuts, failing to measure genuine
  vision-centric reasoning. To address this, we introduce VisuLogic: a benchmark
  of 1,000 human-verified problems across six categories (e.g., quantitative
  shifts, spatial relations, attribute comparisons). These various types of
  questions can be evaluated to assess the visual reasoning capabilities of MLLMs
  from multiple perspectives. We evaluate leading MLLMs on this benchmark and
  analyze their results to identify common failure modes. Most models score below
  30% accuracy-only slightly above the 25% random baseline and far below the
  51.4% achieved by humans-revealing significant gaps in visual reasoning.
  Furthermore, we provide a supplementary training dataset and a
  reinforcement-learning baseline to support further progress.
paper_ai_summary: VisuLogic is a benchmark to evaluate genuine vision-centric reasoning in multimodal large language models, revealing significant performance gaps compared to human accuracy.
paper_ai_keywords: multimodal large language models visual reasoning human-verified problems spatial relations quantitative shifts attribute comparisons reinforcement-learning baseline