SpatialScore: Towards Unified Evaluation for Multimodal Spatial
  Understanding
Multimodal large language models (MLLMs) have achieved impressive success in
question-answering tasks, yet their capabilities for spatial understanding are
less explored. This work investigates a critical question: do existing MLLMs
possess 3D spatial perception and understanding abilities? Concretely, we make
the following contributions in this paper: (i) we introduce VGBench, a
benchmark specifically designed to assess MLLMs for visual geometry perception,
e.g., camera pose and motion estimation; (ii) we propose SpatialScore, the most
comprehensive and diverse multimodal spatial understanding benchmark to date,
integrating VGBench with relevant data from the other 11 existing datasets.
This benchmark comprises 28K samples across various spatial understanding
tasks, modalities, and QA formats, along with a carefully curated challenging
subset, SpatialScore-Hard; (iii) we develop SpatialAgent, a novel multi-agent
system incorporating 9 specialized tools for spatial understanding, supporting
both Plan-Execute and ReAct reasoning paradigms; (iv) we conduct extensive
evaluations to reveal persistent challenges in spatial reasoning while
demonstrating the effectiveness of SpatialAgent. We believe SpatialScore will
offer valuable insights and serve as a rigorous benchmark for the next
evolution of MLLMs.
682fde942f8f73559fcbc5da
Haoning Wu
claimed_verified
2025-05-26T08:16:33.137Z
632c7a0d1d303f5f9acf01b8
https://cdn-avatars.huggingface.co/v1/production/uploads/632c7a0d1d303f5f9acf01b8/T010IFuCp6UaOeIyWhbCk.jpeg
Haoning Wu
user
haoningwu
682fde942f8f73559fcbc5db
Xiao Huang
682fde942f8f73559fcbc5dc
Yaohui Chen
claimed_verified
2025-05-28T09:01:14.383Z
662377137a1a338358811138
/avatars/f6cac3dc1352a05618bcc1eb168b3ef4.svg
Yaohui Chen
user
YaohuiChen
682fde942f8f73559fcbc5dd
Ya Zhang
682fde942f8f73559fcbc5de
Yanfeng Wang
682fde942f8f73559fcbc5df
Weidi Xie
SpatialScore: Towards Unified Evaluation for Multimodal Spatial
  Understanding
Multimodal large language models (MLLMs) have achieved impressive success in
question-answering tasks, yet their capabilities for spatial understanding are
less explored. This work investigates a critical question: do existing MLLMs
possess 3D spatial perception and understanding abilities? Concretely, we make
the following contributions in this paper: (i) we introduce VGBench, a
benchmark specifically designed to assess MLLMs for visual geometry perception,
e.g., camera pose and motion estimation; (ii) we propose SpatialScore, the most
comprehensive and diverse multimodal spatial understanding benchmark to date,
integrating VGBench with relevant data from the other 11 existing datasets.
This benchmark comprises 28K samples across various spatial understanding
tasks, modalities, and QA formats, along with a carefully curated challenging
subset, SpatialScore-Hard; (iii) we develop SpatialAgent, a novel multi-agent
system incorporating 9 specialized tools for spatial understanding, supporting
both Plan-Execute and ReAct reasoning paradigms; (iv) we conduct extensive
evaluations to reveal persistent challenges in spatial reasoning while
demonstrating the effectiveness of SpatialAgent. We believe SpatialScore will
offer valuable insights and serve as a rigorous benchmark for the next
evolution of MLLMs.
SpatialScore benchmarks multimodal large language models for 3D spatial understanding, revealing challenges and showcasing the effectiveness of SpatialAgent with specialized tools.
Multimodal large language models MLLMs VGBench SpatialScore spatial understanding visual geometry perception camera pose motion estimation multi-agent system SpatialAgent Plan-Execute ReAct reasoning paradigms