title: ComboVerse: Compositional 3D Assets Creation Using Spatially-Aware
    Diffusion Guidance
summary: Generating high-quality 3D assets from a given image is highly desirable in
  various applications such as AR/VR. Recent advances in single-image 3D
  generation explore feed-forward models that learn to infer the 3D model of an
  object without optimization. Though promising results have been achieved in
  single object generation, these methods often struggle to model complex 3D
  assets that inherently contain multiple objects. In this work, we present
  ComboVerse, a 3D generation framework that produces high-quality 3D assets with
  complex compositions by learning to combine multiple models. 1) We first
  perform an in-depth analysis of this ``multi-object gap'' from both model and
  data perspectives. 2) Next, with reconstructed 3D models of different objects,
  we seek to adjust their sizes, rotation angles, and locations to create a 3D
  asset that matches the given image. 3) To automate this process, we apply
  spatially-aware score distillation sampling (SSDS) from pretrained diffusion
  models to guide the positioning of objects. Our proposed framework emphasizes
  spatial alignment of objects, compared with standard score distillation
  sampling, and thus achieves more accurate results. Extensive experiments
  validate ComboVerse achieves clear improvements over existing methods in
  generating compositional 3D assets.
paper_authors._id: 65fa45725355a52c782a6e9d
paper_authors.name: Yongwei Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-26T21:21:43.406Z
paper_authors._id: 6458c809b1202755ec5cf33a
paper_authors.avatarUrl: /avatars/0cd92dbb4e85484926c65e5582d89e55.svg
paper_authors.fullname: cyw-3d
paper_authors.type: user
paper_authors.user: cyw-3d
paper_authors._id: 65fa45725355a52c782a6e9e
paper_authors.name: Tengfei Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T13:04:09.578Z
paper_authors._id: 62e7c26236a8e8a827ff0891
paper_authors.avatarUrl: /avatars/15648407233bf5757262dbc04464d02e.svg
paper_authors.fullname: Tengfei Wang
paper_authors.type: user
paper_authors.user: tfwang
paper_authors._id: 65fa45725355a52c782a6e9f
paper_authors.name: Tong Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-11T09:24:40.770Z
paper_authors._id: 6310929fda0aaad2f70397a1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6310929fda0aaad2f70397a1/YoHeYZezmCOXwL-qsiZOc.png
paper_authors.fullname: Tong Wu
paper_authors.type: user
paper_authors.user: wutong16
paper_authors._id: 65fa45725355a52c782a6ea0
paper_authors.name: Xingang Pan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T13:05:04.811Z
paper_authors._id: 646758d6a9b4610868a138ff
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/646758d6a9b4610868a138ff/fNBlK5g4g7vlPAR1bQimZ.png
paper_authors.fullname: Xingang Pan
paper_authors.type: user
paper_authors.user: Xingang-Pan
paper_authors._id: 65fa45725355a52c782a6ea1
paper_authors.name: Kui Jia
paper_authors._id: 65fa45725355a52c782a6ea2
paper_authors.name: Ziwei Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T13:05:19.972Z
paper_authors._id: 62ab1ac1d48b4d8b048a3473
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
paper_authors.fullname: Ziwei Liu
paper_authors.type: user
paper_authors.user: liuziwei7
paper_title: ComboVerse: Compositional 3D Assets Creation Using Spatially-Aware
    Diffusion Guidance
paper_summary: Generating high-quality 3D assets from a given image is highly desirable in
  various applications such as AR/VR. Recent advances in single-image 3D
  generation explore feed-forward models that learn to infer the 3D model of an
  object without optimization. Though promising results have been achieved in
  single object generation, these methods often struggle to model complex 3D
  assets that inherently contain multiple objects. In this work, we present
  ComboVerse, a 3D generation framework that produces high-quality 3D assets with
  complex compositions by learning to combine multiple models. 1) We first
  perform an in-depth analysis of this ``multi-object gap'' from both model and
  data perspectives. 2) Next, with reconstructed 3D models of different objects,
  we seek to adjust their sizes, rotation angles, and locations to create a 3D
  asset that matches the given image. 3) To automate this process, we apply
  spatially-aware score distillation sampling (SSDS) from pretrained diffusion
  models to guide the positioning of objects. Our proposed framework emphasizes
  spatial alignment of objects, compared with standard score distillation
  sampling, and thus achieves more accurate results. Extensive experiments
  validate ComboVerse achieves clear improvements over existing methods in
  generating compositional 3D assets.
paper_ai_keywords: feed-forward models 3D generation single-image 3D generation pretrained diffusion models spatially-aware score distillation sampling (SSDS) spatial alignment