title: Global-Local Tree Search for Language Guided 3D Scene Generation
summary: Large Vision-Language Models (VLMs), such as GPT-4, have achieved remarkable
  success across various fields. However, there are few studies on 3D indoor
  scene generation with VLMs. This paper considers this task as a planning
  problem subject to spatial and layout common sense constraints. To solve the
  problem with a VLM, we propose a new global-local tree search algorithm.
  Globally, the method places each object sequentially and explores multiple
  placements during each placement process, where the problem space is
  represented as a tree. To reduce the depth of the tree, we decompose the scene
  structure hierarchically, i.e. room level, region level, floor object level,
  and supported object level. The algorithm independently generates the floor
  objects in different regions and supported objects placed on different floor
  objects. Locally, we also decompose the sub-task, the placement of each object,
  into multiple steps. The algorithm searches the tree of problem space. To
  leverage the VLM model to produce positions of objects, we discretize the
  top-down view space as a dense grid and fill each cell with diverse emojis to
  make to cells distinct. We prompt the VLM with the emoji grid and the VLM
  produces a reasonable location for the object by describing the position with
  the name of emojis. The quantitative and qualitative experimental results
  illustrate our approach generates more plausible 3D scenes than
  state-of-the-art approaches. Our source code is available at
  https://github.com/dw-dengwei/TreeSearchGen .
paper_authors._id: 67e29a0af38c0d7de45c28ee
paper_authors.name: Wei Deng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-25T12:18:58.901Z
paper_authors._id: 65816205e6b9dfd2cd2cff14
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/yfzfGuD0PRYiumnI-Ei3-.jpeg
paper_authors.fullname: Wei
paper_authors.type: user
paper_authors.user: WeiDeng1999
paper_authors._id: 67e29a0af38c0d7de45c28ef
paper_authors.name: Mengshi Qi
paper_authors._id: 67e29a0af38c0d7de45c28f0
paper_authors.name: Huadong Ma
paper_title: Global-Local Tree Search for Language Guided 3D Scene Generation
paper_summary: Large Vision-Language Models (VLMs), such as GPT-4, have achieved remarkable
  success across various fields. However, there are few studies on 3D indoor
  scene generation with VLMs. This paper considers this task as a planning
  problem subject to spatial and layout common sense constraints. To solve the
  problem with a VLM, we propose a new global-local tree search algorithm.
  Globally, the method places each object sequentially and explores multiple
  placements during each placement process, where the problem space is
  represented as a tree. To reduce the depth of the tree, we decompose the scene
  structure hierarchically, i.e. room level, region level, floor object level,
  and supported object level. The algorithm independently generates the floor
  objects in different regions and supported objects placed on different floor
  objects. Locally, we also decompose the sub-task, the placement of each object,
  into multiple steps. The algorithm searches the tree of problem space. To
  leverage the VLM model to produce positions of objects, we discretize the
  top-down view space as a dense grid and fill each cell with diverse emojis to
  make to cells distinct. We prompt the VLM with the emoji grid and the VLM
  produces a reasonable location for the object by describing the position with
  the name of emojis. The quantitative and qualitative experimental results
  illustrate our approach generates more plausible 3D scenes than
  state-of-the-art approaches. Our source code is available at
  https://github.com/dw-dengwei/TreeSearchGen .
paper_ai_summary: A novel global-local tree search algorithm leverages large vision-language models to generate plausible 3D indoor scenes by decomposing the scene into hierarchical levels and using emojis to guide object placement.
paper_ai_keywords: VLMs GPT-4 3D indoor scene generation global-local tree search algorithm spatial constraints layout common sense constraints scene structure hierarchy room level region level floor object level supported object level top-down view space emoji grid