title: Disentangled 3D Scene Generation with Layout Learning
summary: We introduce a method to generate 3D scenes that are disentangled into their
  component objects. This disentanglement is unsupervised, relying only on the
  knowledge of a large pretrained text-to-image model. Our key insight is that
  objects can be discovered by finding parts of a 3D scene that, when rearranged
  spatially, still produce valid configurations of the same scene. Concretely,
  our method jointly optimizes multiple NeRFs from scratch - each representing
  its own object - along with a set of layouts that composite these objects into
  scenes. We then encourage these composited scenes to be in-distribution
  according to the image generator. We show that despite its simplicity, our
  approach successfully generates 3D scenes decomposed into individual objects,
  enabling new capabilities in text-to-3D content creation. For results and an
  interactive demo, see our project page at https://dave.ml/layoutlearning/
paper_authors._id: 65deb5036d161782405dd172
paper_authors.name: Dave Epstein
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-28T12:31:31.787Z
paper_authors._id: 64caa93391726743360ab1f0
paper_authors.avatarUrl: /avatars/992cb5aaf42d21be03a998656bb485d7.svg
paper_authors.fullname: Dave Epstein
paper_authors.type: user
paper_authors.user: daveepstein
paper_authors._id: 65deb5036d161782405dd173
paper_authors.name: Ben Poole
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-28T12:31:37.752Z
paper_authors._id: 63213768d2d45f315183abc3
paper_authors.avatarUrl: /avatars/521f52f430b73e514afb06e3a9d6779d.svg
paper_authors.fullname: Ben Poole
paper_authors.type: user
paper_authors.user: doinkda
paper_authors._id: 65deb5036d161782405dd174
paper_authors.name: Ben Mildenhall
paper_authors._id: 65deb5036d161782405dd175
paper_authors.name: Alexei A. Efros
paper_authors._id: 65deb5036d161782405dd176
paper_authors.name: Aleksander Holynski
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-28T12:31:57.221Z
paper_authors._id: 62f6e2792e53c2efd33faa92
paper_authors.avatarUrl: /avatars/85d94b4022577747b8d2d10a82c2f3c7.svg
paper_authors.fullname: Aleksander Holynski
paper_authors.type: user
paper_authors.user: holynski
paper_title: Disentangled 3D Scene Generation with Layout Learning
paper_summary: We introduce a method to generate 3D scenes that are disentangled into their
  component objects. This disentanglement is unsupervised, relying only on the
  knowledge of a large pretrained text-to-image model. Our key insight is that
  objects can be discovered by finding parts of a 3D scene that, when rearranged
  spatially, still produce valid configurations of the same scene. Concretely,
  our method jointly optimizes multiple NeRFs from scratch - each representing
  its own object - along with a set of layouts that composite these objects into
  scenes. We then encourage these composited scenes to be in-distribution
  according to the image generator. We show that despite its simplicity, our
  approach successfully generates 3D scenes decomposed into individual objects,
  enabling new capabilities in text-to-3D content creation. For results and an
  interactive demo, see our project page at https://dave.ml/layoutlearning/
paper_ai_summary: Unsupervised disentanglement of 3D scenes into component objects using pretrained text-to-image models and joint optimization of NeRFs and layouts.
paper_ai_keywords: NeRFs in-distribution text-to-3D scene decomposition object discovery