title: Seeing from Another Perspective: Evaluating Multi-View Understanding in
    MLLMs
summary: Multi-view understanding, the ability to reconcile visual information across
  diverse viewpoints for effective navigation, manipulation, and 3D scene
  comprehension, is a fundamental challenge in Multi-Modal Large Language Models
  (MLLMs) to be used as embodied agents. While recent MLLMs have shown impressive
  advances in high-level reasoning and planning, they frequently fall short when
  confronted with multi-view geometric consistency and cross-view correspondence.
  To comprehensively evaluate the challenges of MLLMs in multi-view scene
  reasoning, we propose All-Angles Bench, a benchmark of over 2,100 human
  carefully annotated multi-view question-answer pairs across 90 diverse
  real-world scenes. Our six tasks (counting, attribute identification, relative
  distance, relative direction, object manipulation, and camera pose estimation)
  specifically test model's geometric correspondence and the capacity to align
  information consistently across views. Our extensive experiments, benchmark on
  27 representative MLLMs including Gemini-2.0-Flash, Claude-3.7-Sonnet, and
  GPT-4o against human evaluators reveals a substantial performance gap,
  indicating that current MLLMs remain far from human-level proficiency. Through
  in-depth analysis, we show that MLLMs are particularly underperforming under
  two aspects: (1) cross-view correspondence for partially occluded views and (2)
  establishing the coarse camera poses. These findings highlight the necessity of
  domain-specific refinements or modules that embed stronger multi-view
  awareness. We believe that our All-Angles Bench offers valuable insights and
  contribute to bridging the gap between MLLMs and human-level multi-view
  understanding. The project and benchmark are publicly available at
  https://danielchyeh.github.io/All-Angles-Bench/.
paper_authors._id: 680703e4c0ce7eea9ba1a548
paper_authors.name: Chun-Hsiao Yeh
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-22T09:50:54.878Z
paper_authors._id: 6499eca0685215f7247bd5ce
paper_authors.avatarUrl: /avatars/b6fea0c33c3c930c7314b99b414072a9.svg
paper_authors.fullname: Chun-Hsiao Yeh
paper_authors.type: user
paper_authors.user: danielchyeh
paper_authors._id: 680703e4c0ce7eea9ba1a549
paper_authors.name: Chenyu Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-22T09:50:52.696Z
paper_authors._id: 641bd539aebaa27e07540613
paper_authors.avatarUrl: /avatars/000e8057e7280b9eba9bec5116d14718.svg
paper_authors.fullname: chenyu wang
paper_authors.type: user
paper_authors.user: ch-chenyu
paper_authors._id: 680703e4c0ce7eea9ba1a54a
paper_authors.name: Shengbang Tong
paper_authors._id: 680703e4c0ce7eea9ba1a54b
paper_authors.name: Ta-Ying Cheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-22T10:58:20.913Z
paper_authors._id: 6442b024e255a338677d78fa
paper_authors.avatarUrl: /avatars/516769c6c3bc7a0d416924181c6866dc.svg
paper_authors.fullname: Ta-Ying Cheng
paper_authors.type: user
paper_authors.user: chengtim
paper_authors._id: 680703e4c0ce7eea9ba1a54c
paper_authors.name: Rouyu Wang
paper_authors._id: 680703e4c0ce7eea9ba1a54d
paper_authors.name: Tianzhe Chu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-28T12:54:14.917Z
paper_authors._id: 65127ec162b7d28b1eaab17a
paper_authors.avatarUrl: /avatars/0bcfeb68a405be4efb6e8a29738a5598.svg
paper_authors.fullname: Tianzhe
paper_authors.type: user
paper_authors.user: tianzhechu
paper_authors._id: 680703e4c0ce7eea9ba1a54e
paper_authors.name: Yuexiang Zhai
paper_authors._id: 680703e4c0ce7eea9ba1a54f
paper_authors.name: Yubei Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-22T10:59:03.407Z
paper_authors._id: 65dd08d38c103eb3d93346bd
paper_authors.avatarUrl: /avatars/6f84dba62d93d21e48212951f54f7e09.svg
paper_authors.fullname: Yubei Chen
paper_authors.type: user
paper_authors.user: yubei
paper_authors._id: 680703e4c0ce7eea9ba1a550
paper_authors.name: Shenghua Gao
paper_authors._id: 680703e4c0ce7eea9ba1a551
paper_authors.name: Yi Ma
paper_title: Seeing from Another Perspective: Evaluating Multi-View Understanding in
    MLLMs
paper_summary: Multi-view understanding, the ability to reconcile visual information across
  diverse viewpoints for effective navigation, manipulation, and 3D scene
  comprehension, is a fundamental challenge in Multi-Modal Large Language Models
  (MLLMs) to be used as embodied agents. While recent MLLMs have shown impressive
  advances in high-level reasoning and planning, they frequently fall short when
  confronted with multi-view geometric consistency and cross-view correspondence.
  To comprehensively evaluate the challenges of MLLMs in multi-view scene
  reasoning, we propose All-Angles Bench, a benchmark of over 2,100 human
  carefully annotated multi-view question-answer pairs across 90 diverse
  real-world scenes. Our six tasks (counting, attribute identification, relative
  distance, relative direction, object manipulation, and camera pose estimation)
  specifically test model's geometric correspondence and the capacity to align
  information consistently across views. Our extensive experiments, benchmark on
  27 representative MLLMs including Gemini-2.0-Flash, Claude-3.7-Sonnet, and
  GPT-4o against human evaluators reveals a substantial performance gap,
  indicating that current MLLMs remain far from human-level proficiency. Through
  in-depth analysis, we show that MLLMs are particularly underperforming under
  two aspects: (1) cross-view correspondence for partially occluded views and (2)
  establishing the coarse camera poses. These findings highlight the necessity of
  domain-specific refinements or modules that embed stronger multi-view
  awareness. We believe that our All-Angles Bench offers valuable insights and
  contribute to bridging the gap between MLLMs and human-level multi-view
  understanding. The project and benchmark are publicly available at
  https://danielchyeh.github.io/All-Angles-Bench/.
paper_ai_summary: All-Angles Bench evaluates the multi-view reasoning capabilities of MLLMs using diverse human-annotated datasets, revealing significant gaps in cross-view correspondence and camera pose estimation.
paper_ai_keywords: Multi-view understanding MLLMs All-Angles Bench multi-view question-answer pairs cross-view correspondence geometric correspondence camera pose estimation