title: IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality
    3D Generation
summary: Most text-to-3D generators build upon off-the-shelf text-to-image models
  trained on billions of images. They use variants of Score Distillation Sampling
  (SDS), which is slow, somewhat unstable, and prone to artifacts. A mitigation
  is to fine-tune the 2D generator to be multi-view aware, which can help
  distillation or can be combined with reconstruction networks to output 3D
  objects directly. In this paper, we further explore the design space of
  text-to-3D models. We significantly improve multi-view generation by
  considering video instead of image generators. Combined with a 3D
  reconstruction algorithm which, by using Gaussian splatting, can optimize a
  robust image-based loss, we directly produce high-quality 3D outputs from the
  generated views. Our new method, IM-3D, reduces the number of evaluations of
  the 2D generator network 10-100x, resulting in a much more efficient pipeline,
  better quality, fewer geometric inconsistencies, and higher yield of usable 3D
  assets.
paper_authors._id: 65cc30057643d0e69bb18894
paper_authors.name: Luke Melas-Kyriazi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-14T11:03:40.117Z
paper_authors._id: 6244593ec661f366527f6a78
paper_authors.avatarUrl: /avatars/4e2cf8894c3268c824160249cdfaf3ab.svg
paper_authors.fullname: Luke Melas-Kyriazi
paper_authors.type: user
paper_authors.user: lukemelas
paper_authors._id: 65cc30057643d0e69bb18895
paper_authors.name: Iro Laina
paper_authors._id: 65cc30057643d0e69bb18896
paper_authors.name: Christian Rupprecht
paper_authors._id: 65cc30057643d0e69bb18897
paper_authors.name: Natalia Neverova
paper_authors._id: 65cc30057643d0e69bb18898
paper_authors.name: Andrea Vedaldi
paper_authors._id: 65cc30057643d0e69bb18899
paper_authors.name: Oran Gafni
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-14T11:04:09.310Z
paper_authors._id: 6345a97d547c70e4b7ce9169
paper_authors.avatarUrl: /avatars/5d233e65eadfa7d9ef0390a584df5451.svg
paper_authors.fullname: Oran Gafni
paper_authors.type: user
paper_authors.user: Oran22
paper_authors._id: 65cc30057643d0e69bb1889a
paper_authors.name: Filippos Kokkinos
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-02-14T03:14:14.497Z
paper_authors._id: 64918e6bd0aa14078dee0929
paper_authors.avatarUrl: /avatars/407d6996d5d297ad3492a9a2382205cd.svg
paper_authors.fullname: Filippos
paper_authors.type: user
paper_authors.user: fkokkinos
paper_title: IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality
    3D Generation
paper_summary: Most text-to-3D generators build upon off-the-shelf text-to-image models
  trained on billions of images. They use variants of Score Distillation Sampling
  (SDS), which is slow, somewhat unstable, and prone to artifacts. A mitigation
  is to fine-tune the 2D generator to be multi-view aware, which can help
  distillation or can be combined with reconstruction networks to output 3D
  objects directly. In this paper, we further explore the design space of
  text-to-3D models. We significantly improve multi-view generation by
  considering video instead of image generators. Combined with a 3D
  reconstruction algorithm which, by using Gaussian splatting, can optimize a
  robust image-based loss, we directly produce high-quality 3D outputs from the
  generated views. Our new method, IM-3D, reduces the number of evaluations of
  the 2D generator network 10-100x, resulting in a much more efficient pipeline,
  better quality, fewer geometric inconsistencies, and higher yield of usable 3D
  assets.
paper_ai_keywords: Score Distillation Sampling (SDS) Gaussian splatting image-based loss IM-3D multi-view generation 3D reconstruction algorithm 2D generator network evaluations geometric inconsistencies 3D assets