title: 3D Question Answering for City Scene Understanding
summary: 3D multimodal question answering (MQA) plays a crucial role in scene
  understanding by enabling intelligent agents to comprehend their surroundings
  in 3D environments. While existing research has primarily focused on indoor
  household tasks and outdoor roadside autonomous driving tasks, there has been
  limited exploration of city-level scene understanding tasks. Furthermore,
  existing research faces challenges in understanding city scenes, due to the
  absence of spatial semantic information and human-environment interaction
  information at the city level.To address these challenges, we investigate 3D
  MQA from both dataset and method perspectives. From the dataset perspective, we
  introduce a novel 3D MQA dataset named City-3DQA for city-level scene
  understanding, which is the first dataset to incorporate scene semantic and
  human-environment interactive tasks within the city. From the method
  perspective, we propose a Scene graph enhanced City-level Understanding method
  (Sg-CityU), which utilizes the scene graph to introduce the spatial semantic. A
  new benchmark is reported and our proposed Sg-CityU achieves accuracy of 63.94
  % and 63.76 % in different settings of City-3DQA. Compared to indoor 3D MQA
  methods and zero-shot using advanced large language models (LLMs), Sg-CityU
  demonstrates state-of-the-art (SOTA) performance in robustness and
  generalization.
paper_authors._id: 66a848766419b5b8fcea0163
paper_authors.name: Penglei Sun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-30T09:01:33.517Z
paper_authors._id: 64eded5fdfe0a679d840bc98
paper_authors.avatarUrl: /avatars/4d4c67c13e547a4d296a301e8694e79e.svg
paper_authors.fullname: sunpenglei
paper_authors.type: user
paper_authors.user: sunpenglei
paper_authors._id: 66a848766419b5b8fcea0164
paper_authors.name: Yaoxian Song
paper_authors._id: 66a848766419b5b8fcea0165
paper_authors.name: Xiang Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-30T07:25:52.460Z
paper_authors._id: 63024676056ec3a2a8714b24
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1661093436322-noauth.jpeg
paper_authors.fullname: Xiang Liu
paper_authors.type: user
paper_authors.user: Dominic789654
paper_authors._id: 66a848766419b5b8fcea0166
paper_authors.name: Xiaofei Yang
paper_authors._id: 66a848766419b5b8fcea0167
paper_authors.name: Qiang Wang
paper_authors._id: 66a848766419b5b8fcea0168
paper_authors.name: Tiefeng Li
paper_authors._id: 66a848766419b5b8fcea0169
paper_authors.name: Yang Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-30T13:46:20.318Z
paper_authors._id: 66a8e0fcb82318f0a13deafc
paper_authors.avatarUrl: /avatars/a6794b5299a5af896b8d9843b20cef9d.svg
paper_authors.fullname: Yang Yang
paper_authors.type: user
paper_authors.user: yyiot
paper_authors._id: 66a848766419b5b8fcea016a
paper_authors.name: Xiaowen Chu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-30T09:03:05.982Z
paper_authors._id: 6676935fcd0b89a0115174b0
paper_authors.avatarUrl: /avatars/4caca1b672d29e787814f9a30bf20bcc.svg
paper_authors.fullname: Xiaowen Chu
paper_authors.type: user
paper_authors.user: wenxinsiju
paper_title: 3D Question Answering for City Scene Understanding
paper_summary: 3D multimodal question answering (MQA) plays a crucial role in scene
  understanding by enabling intelligent agents to comprehend their surroundings
  in 3D environments. While existing research has primarily focused on indoor
  household tasks and outdoor roadside autonomous driving tasks, there has been
  limited exploration of city-level scene understanding tasks. Furthermore,
  existing research faces challenges in understanding city scenes, due to the
  absence of spatial semantic information and human-environment interaction
  information at the city level.To address these challenges, we investigate 3D
  MQA from both dataset and method perspectives. From the dataset perspective, we
  introduce a novel 3D MQA dataset named City-3DQA for city-level scene
  understanding, which is the first dataset to incorporate scene semantic and
  human-environment interactive tasks within the city. From the method
  perspective, we propose a Scene graph enhanced City-level Understanding method
  (Sg-CityU), which utilizes the scene graph to introduce the spatial semantic. A
  new benchmark is reported and our proposed Sg-CityU achieves accuracy of 63.94
  % and 63.76 % in different settings of City-3DQA. Compared to indoor 3D MQA
  methods and zero-shot using advanced large language models (LLMs), Sg-CityU
  demonstrates state-of-the-art (SOTA) performance in robustness and
  generalization.
paper_ai_summary: The research introduces a 3D multimodal question answering dataset and method for city-level scene understanding, incorporating spatial semantics and human-environment interactions, achieving state-of-the-art performance.
paper_ai_keywords: 3D multimodal question answering 3D MQA city-level scene understanding scene graph City-3DQA Scene graph enhanced City-level Understanding Sg-CityU