title: FirePlace: Geometric Refinements of LLM Common Sense Reasoning for 3D
    Object Placement
summary: Scene generation with 3D assets presents a complex challenge, requiring both
  high-level semantic understanding and low-level geometric reasoning. While
  Multimodal Large Language Models (MLLMs) excel at semantic tasks, their
  application to 3D scene generation is hindered by their limited grounding on 3D
  geometry. In this paper, we investigate how to best work with MLLMs in an
  object placement task. Towards this goal, we introduce a novel framework,
  FirePlace, that applies existing MLLMs in (1) 3D geometric reasoning and the
  extraction of relevant geometric details from the 3D scene, (2) constructing
  and solving geometric constraints on the extracted low-level geometry, and (3)
  pruning for final placements that conform to common sense. By combining
  geometric reasoning with real-world understanding of MLLMs, our method can
  propose object placements that satisfy both geometric constraints as well as
  high-level semantic common-sense considerations. Our experiments show that
  these capabilities allow our method to place objects more effectively in
  complex scenes with intricate geometry, surpassing the quality of prior work.
paper_authors._id: 67e480296d47bc4578afc3a4
paper_authors.name: Ian Huang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-27T09:03:43.079Z
paper_authors._id: 639ce51c66106be14370f510
paper_authors.avatarUrl: /avatars/74f68f5951fff9c02c7607ff197812aa.svg
paper_authors.fullname: Ian Huang
paper_authors.type: user
paper_authors.user: ianhuang
paper_authors._id: 67e480296d47bc4578afc3a5
paper_authors.name: Yanan Bao
paper_authors._id: 67e480296d47bc4578afc3a6
paper_authors.name: Karen Truong
paper_authors._id: 67e480296d47bc4578afc3a7
paper_authors.name: Howard Zhou
paper_authors._id: 67e480296d47bc4578afc3a8
paper_authors.name: Cordelia Schmid
paper_authors._id: 67e480296d47bc4578afc3a9
paper_authors.name: Leonidas Guibas
paper_authors._id: 67e480296d47bc4578afc3aa
paper_authors.name: Alireza Fathi
paper_title: FirePlace: Geometric Refinements of LLM Common Sense Reasoning for 3D
    Object Placement
paper_summary: Scene generation with 3D assets presents a complex challenge, requiring both
  high-level semantic understanding and low-level geometric reasoning. While
  Multimodal Large Language Models (MLLMs) excel at semantic tasks, their
  application to 3D scene generation is hindered by their limited grounding on 3D
  geometry. In this paper, we investigate how to best work with MLLMs in an
  object placement task. Towards this goal, we introduce a novel framework,
  FirePlace, that applies existing MLLMs in (1) 3D geometric reasoning and the
  extraction of relevant geometric details from the 3D scene, (2) constructing
  and solving geometric constraints on the extracted low-level geometry, and (3)
  pruning for final placements that conform to common sense. By combining
  geometric reasoning with real-world understanding of MLLMs, our method can
  propose object placements that satisfy both geometric constraints as well as
  high-level semantic common-sense considerations. Our experiments show that
  these capabilities allow our method to place objects more effectively in
  complex scenes with intricate geometry, surpassing the quality of prior work.
paper_ai_summary: FirePlace combines Multimodal Large Language Models with geometric reasoning to effectively place objects in complex 3D scenes, outperforming existing methods.
paper_ai_keywords: Multimodal Large Language Models 3D geometric reasoning object placement geometric constraints common sense scene generation