title: Spatial Knowledge Graph-Guided Multimodal Synthesis
summary: Recent advances in multimodal large language models (MLLMs) have
  significantly enhanced their capabilities; however, their spatial perception
  abilities remain a notable limitation. To address this challenge, multimodal
  data synthesis offers a promising solution. Yet, ensuring that synthesized data
  adhere to spatial common sense is a non-trivial task. In this work, we
  introduce SKG2Data, a novel multimodal synthesis approach guided by spatial
  knowledge graphs, grounded in the concept of knowledge-to-data generation.
  SKG2Data automatically constructs a Spatial Knowledge Graph (SKG) to emulate
  human-like perception of spatial directions and distances, which is
  subsequently utilized to guide multimodal data synthesis. Extensive experiments
  demonstrate that data synthesized from diverse types of spatial knowledge,
  including direction and distance, not only enhance the spatial perception and
  reasoning abilities of MLLMs but also exhibit strong generalization
  capabilities. We hope that the idea of knowledge-based data synthesis can
  advance the development of spatial intelligence.
paper_authors._id: 6837b67765209bfdd650594e
paper_authors.name: Yida Xue
paper_authors._id: 6837b67765209bfdd650594f
paper_authors.name: Zhen Bi
paper_authors._id: 6837b67765209bfdd6505950
paper_authors.name: Jinnan Yang
paper_authors._id: 6837b67765209bfdd6505951
paper_authors.name: Jungang Lou
paper_authors._id: 6837b67765209bfdd6505952
paper_authors.name: Huajun Chen
paper_authors._id: 6837b67765209bfdd6505953
paper_authors.name: Ningyu Zhang
paper_title: Spatial Knowledge Graph-Guided Multimodal Synthesis
paper_summary: Recent advances in multimodal large language models (MLLMs) have
  significantly enhanced their capabilities; however, their spatial perception
  abilities remain a notable limitation. To address this challenge, multimodal
  data synthesis offers a promising solution. Yet, ensuring that synthesized data
  adhere to spatial common sense is a non-trivial task. In this work, we
  introduce SKG2Data, a novel multimodal synthesis approach guided by spatial
  knowledge graphs, grounded in the concept of knowledge-to-data generation.
  SKG2Data automatically constructs a Spatial Knowledge Graph (SKG) to emulate
  human-like perception of spatial directions and distances, which is
  subsequently utilized to guide multimodal data synthesis. Extensive experiments
  demonstrate that data synthesized from diverse types of spatial knowledge,
  including direction and distance, not only enhance the spatial perception and
  reasoning abilities of MLLMs but also exhibit strong generalization
  capabilities. We hope that the idea of knowledge-based data synthesis can
  advance the development of spatial intelligence.
paper_ai_summary: SKG2Data enhances spatial perception and reasoning in multimodal large language models by using spatial knowledge graphs to guide data synthesis.
paper_ai_keywords: multimodal large language models spatial perception multimodal synthesis spatial knowledge graphs knowledge-to-data generation spatial directions spatial distances spatial intelligence