Seeing from Another Perspective: Evaluating Multi-View Understanding in
  MLLMs
Multi-view understanding, the ability to reconcile visual information across
diverse viewpoints for effective navigation, manipulation, and 3D scene
comprehension, is a fundamental challenge in Multi-Modal Large Language Models
(MLLMs) to be used as embodied agents. While recent MLLMs have shown impressive
advances in high-level reasoning and planning, they frequently fall short when
confronted with multi-view geometric consistency and cross-view correspondence.
To comprehensively evaluate the challenges of MLLMs in multi-view scene
reasoning, we propose All-Angles Bench, a benchmark of over 2,100 human
carefully annotated multi-view question-answer pairs across 90 diverse
real-world scenes. Our six tasks (counting, attribute identification, relative
distance, relative direction, object manipulation, and camera pose estimation)
specifically test model's geometric correspondence and the capacity to align
information consistently across views. Our extensive experiments, benchmark on
27 representative MLLMs including Gemini-2.0-Flash, Claude-3.7-Sonnet, and
GPT-4o against human evaluators reveals a substantial performance gap,
indicating that current MLLMs remain far from human-level proficiency. Through
in-depth analysis, we show that MLLMs are particularly underperforming under
two aspects: (1) cross-view correspondence for partially occluded views and (2)
establishing the coarse camera poses. These findings highlight the necessity of
domain-specific refinements or modules that embed stronger multi-view
awareness. We believe that our All-Angles Bench offers valuable insights and
contribute to bridging the gap between MLLMs and human-level multi-view
understanding. The project and benchmark are publicly available at
https://danielchyeh.github.io/All-Angles-Bench/.
680703e4c0ce7eea9ba1a548
Chun-Hsiao Yeh
claimed_verified
2025-04-22T09:50:54.878Z
6499eca0685215f7247bd5ce
/avatars/b6fea0c33c3c930c7314b99b414072a9.svg
Chun-Hsiao Yeh
user
danielchyeh
680703e4c0ce7eea9ba1a549
Chenyu Wang
claimed_verified
2025-04-22T09:50:52.696Z
641bd539aebaa27e07540613
/avatars/000e8057e7280b9eba9bec5116d14718.svg
chenyu wang
user
ch-chenyu
680703e4c0ce7eea9ba1a54a
Shengbang Tong
680703e4c0ce7eea9ba1a54b
Ta-Ying Cheng
admin_assigned
2025-04-22T10:58:20.913Z
6442b024e255a338677d78fa
/avatars/516769c6c3bc7a0d416924181c6866dc.svg
Ta-Ying Cheng
user
chengtim
680703e4c0ce7eea9ba1a54c
Rouyu Wang
680703e4c0ce7eea9ba1a54d
Tianzhe Chu
claimed_verified
2025-04-28T12:54:14.917Z
65127ec162b7d28b1eaab17a
/avatars/0bcfeb68a405be4efb6e8a29738a5598.svg
Tianzhe
user
tianzhechu
680703e4c0ce7eea9ba1a54e
Yuexiang Zhai
680703e4c0ce7eea9ba1a54f
Yubei Chen
admin_assigned
2025-04-22T10:59:03.407Z
65dd08d38c103eb3d93346bd
/avatars/6f84dba62d93d21e48212951f54f7e09.svg
Yubei Chen
user
yubei
680703e4c0ce7eea9ba1a550
Shenghua Gao
680703e4c0ce7eea9ba1a551
Yi Ma
Seeing from Another Perspective: Evaluating Multi-View Understanding in
  MLLMs
Multi-view understanding, the ability to reconcile visual information across
diverse viewpoints for effective navigation, manipulation, and 3D scene
comprehension, is a fundamental challenge in Multi-Modal Large Language Models
(MLLMs) to be used as embodied agents. While recent MLLMs have shown impressive
advances in high-level reasoning and planning, they frequently fall short when
confronted with multi-view geometric consistency and cross-view correspondence.
To comprehensively evaluate the challenges of MLLMs in multi-view scene
reasoning, we propose All-Angles Bench, a benchmark of over 2,100 human
carefully annotated multi-view question-answer pairs across 90 diverse
real-world scenes. Our six tasks (counting, attribute identification, relative
distance, relative direction, object manipulation, and camera pose estimation)
specifically test model's geometric correspondence and the capacity to align
information consistently across views. Our extensive experiments, benchmark on
27 representative MLLMs including Gemini-2.0-Flash, Claude-3.7-Sonnet, and
GPT-4o against human evaluators reveals a substantial performance gap,
indicating that current MLLMs remain far from human-level proficiency. Through
in-depth analysis, we show that MLLMs are particularly underperforming under
two aspects: (1) cross-view correspondence for partially occluded views and (2)
establishing the coarse camera poses. These findings highlight the necessity of
domain-specific refinements or modules that embed stronger multi-view
awareness. We believe that our All-Angles Bench offers valuable insights and
contribute to bridging the gap between MLLMs and human-level multi-view
understanding. The project and benchmark are publicly available at
https://danielchyeh.github.io/All-Angles-Bench/.
All-Angles Bench evaluates the multi-view reasoning capabilities of MLLMs using diverse human-annotated datasets, revealing significant gaps in cross-view correspondence and camera pose estimation.
Multi-view understanding MLLMs All-Angles Bench multi-view question-answer pairs cross-view correspondence geometric correspondence camera pose estimation