title: OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for
    Vision Language Models
summary: Spatial reasoning is a key aspect of cognitive psychology and remains a major
  bottleneck for current vision-language models (VLMs). While extensive research
  has aimed to evaluate or improve VLMs' understanding of basic spatial
  relations, such as distinguishing left from right, near from far, and object
  counting, these tasks represent only the most fundamental level of spatial
  reasoning. In this work, we introduce OmniSpatial, a comprehensive and
  challenging benchmark for spatial reasoning, grounded in cognitive psychology.
  OmniSpatial covers four major categories: dynamic reasoning, complex spatial
  logic, spatial interaction, and perspective-taking, with 50 fine-grained
  subcategories. Through Internet data crawling and careful manual annotation, we
  construct over 1.5K question-answer pairs. Extensive experiments show that both
  open- and closed-source VLMs, as well as existing reasoning and spatial
  understanding models, exhibit significant limitations in comprehensive spatial
  understanding. We further analyze failure cases and propose potential
  directions for future research.
paper_authors._id: 683fe55868402c738a8e5ee4
paper_authors.name: Mengdi Jia
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T07:28:05.023Z
paper_authors._id: 6774dbb9607615a0ebfd21e9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6774dbb9607615a0ebfd21e9/L2JAcfpqjy7ixJidwy9zy.jpeg
paper_authors.fullname: Mengdi Jia
paper_authors.type: user
paper_authors.user: MengdiJia
paper_authors._id: 683fe55868402c738a8e5ee5
paper_authors.name: Zekun Qi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-04T08:54:22.042Z
paper_authors._id: 63c3e8abc7d7f4c63a515a02
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63c3e8abc7d7f4c63a515a02/npMHnVP2hHLbvoUGe7C4O.jpeg
paper_authors.fullname: Zekun Qi
paper_authors.type: user
paper_authors.user: qizekun
paper_authors._id: 683fe55868402c738a8e5ee6
paper_authors.name: Shaochen Zhang
paper_authors._id: 683fe55868402c738a8e5ee7
paper_authors.name: Wenyao Zhang
paper_authors._id: 683fe55868402c738a8e5ee8
paper_authors.name: Xinqiang Yu
paper_authors._id: 683fe55868402c738a8e5ee9
paper_authors.name: Jiawei He
paper_authors._id: 683fe55868402c738a8e5eea
paper_authors.name: He Wang
paper_authors._id: 683fe55868402c738a8e5eeb
paper_authors.name: Li Yi
paper_title: OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for
    Vision Language Models
paper_summary: Spatial reasoning is a key aspect of cognitive psychology and remains a major
  bottleneck for current vision-language models (VLMs). While extensive research
  has aimed to evaluate or improve VLMs' understanding of basic spatial
  relations, such as distinguishing left from right, near from far, and object
  counting, these tasks represent only the most fundamental level of spatial
  reasoning. In this work, we introduce OmniSpatial, a comprehensive and
  challenging benchmark for spatial reasoning, grounded in cognitive psychology.
  OmniSpatial covers four major categories: dynamic reasoning, complex spatial
  logic, spatial interaction, and perspective-taking, with 50 fine-grained
  subcategories. Through Internet data crawling and careful manual annotation, we
  construct over 1.5K question-answer pairs. Extensive experiments show that both
  open- and closed-source VLMs, as well as existing reasoning and spatial
  understanding models, exhibit significant limitations in comprehensive spatial
  understanding. We further analyze failure cases and propose potential
  directions for future research.
paper_ai_summary: A comprehensive benchmark called OmniSpatial evaluates vision-language models' understanding of advanced spatial reasoning tasks, revealing significant limitations across various models.
paper_ai_keywords: vision-language models spatial reasoning cognitive psychology dynamic reasoning complex spatial logic spatial interaction perspective-taking question-answer pairs