title: Generic 3D Diffusion Adapter Using Controlled Multi-View Editing
summary: Open-domain 3D object synthesis has been lagging behind image synthesis due
  to limited data and higher computational complexity. To bridge this gap, recent
  works have investigated multi-view diffusion but often fall short in either 3D
  consistency, visual quality, or efficiency. This paper proposes MVEdit, which
  functions as a 3D counterpart of SDEdit, employing ancestral sampling to
  jointly denoise multi-view images and output high-quality textured meshes.
  Built on off-the-shelf 2D diffusion models, MVEdit achieves 3D consistency
  through a training-free 3D Adapter, which lifts the 2D views of the last
  timestep into a coherent 3D representation, then conditions the 2D views of the
  next timestep using rendered views, without uncompromising visual quality. With
  an inference time of only 2-5 minutes, this framework achieves better trade-off
  between quality and speed than score distillation. MVEdit is highly versatile
  and extendable, with a wide range of applications including text/image-to-3D
  generation, 3D-to-3D editing, and high-quality texture synthesis. In
  particular, evaluations demonstrate state-of-the-art performance in both
  image-to-3D and text-guided texture generation tasks. Additionally, we
  introduce a method for fine-tuning 2D latent diffusion models on small 3D
  datasets with limited resources, enabling fast low-resolution text-to-3D
  initialization.
paper_authors._id: 65f90f3248699f59af9f97fe
paper_authors.name: Hansheng Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-19T08:45:21.231Z
paper_authors._id: 638067fcb334960c987fbeda
paper_authors.avatarUrl: /avatars/87f1eaaf6b3a9c0d47d6f406261ccc18.svg
paper_authors.fullname: Hansheng Chen
paper_authors.type: user
paper_authors.user: Lakonik
paper_authors._id: 65f90f3248699f59af9f97ff
paper_authors.name: Ruoxi Shi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-19T09:44:19.854Z
paper_authors._id: 640820798dca6cec91cb2f0a
paper_authors.avatarUrl: /avatars/e011bc04facd251570247d1ae214dbde.svg
paper_authors.fullname: Ruoxi Shi
paper_authors.type: user
paper_authors.user: eliphatfs
paper_authors._id: 65f90f3248699f59af9f9800
paper_authors.name: Yulin Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-19T09:44:38.242Z
paper_authors._id: 647169fb1e9659ae5c947705
paper_authors.avatarUrl: /avatars/d19415d746af1d48ccbf5aafb84153c1.svg
paper_authors.fullname: Yulin Liu
paper_authors.type: user
paper_authors.user: yulinliu
paper_authors._id: 65f90f3248699f59af9f9801
paper_authors.name: Bokui Shen
paper_authors._id: 65f90f3248699f59af9f9802
paper_authors.name: Jiayuan Gu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-19T09:44:56.373Z
paper_authors._id: 638aa283cebef0d13aa2ec2e
paper_authors.avatarUrl: /avatars/3b67e6a6073033864a817230e97c27ca.svg
paper_authors.fullname: Jiayuan Gu
paper_authors.type: user
paper_authors.user: jigu
paper_authors._id: 65f90f3248699f59af9f9803
paper_authors.name: Gordon Wetzstein
paper_authors._id: 65f90f3248699f59af9f9804
paper_authors.name: Hao Su
paper_authors._id: 65f90f3248699f59af9f9805
paper_authors.name: Leonidas Guibas
paper_title: Generic 3D Diffusion Adapter Using Controlled Multi-View Editing
paper_summary: Open-domain 3D object synthesis has been lagging behind image synthesis due
  to limited data and higher computational complexity. To bridge this gap, recent
  works have investigated multi-view diffusion but often fall short in either 3D
  consistency, visual quality, or efficiency. This paper proposes MVEdit, which
  functions as a 3D counterpart of SDEdit, employing ancestral sampling to
  jointly denoise multi-view images and output high-quality textured meshes.
  Built on off-the-shelf 2D diffusion models, MVEdit achieves 3D consistency
  through a training-free 3D Adapter, which lifts the 2D views of the last
  timestep into a coherent 3D representation, then conditions the 2D views of the
  next timestep using rendered views, without uncompromising visual quality. With
  an inference time of only 2-5 minutes, this framework achieves better trade-off
  between quality and speed than score distillation. MVEdit is highly versatile
  and extendable, with a wide range of applications including text/image-to-3D
  generation, 3D-to-3D editing, and high-quality texture synthesis. In
  particular, evaluations demonstrate state-of-the-art performance in both
  image-to-3D and text-guided texture generation tasks. Additionally, we
  introduce a method for fine-tuning 2D latent diffusion models on small 3D
  datasets with limited resources, enabling fast low-resolution text-to-3D
  initialization.
paper_ai_summary: MVEdit uses ancestral sampling to denoise multi-view images and generate 3D textured meshes, achieving high-quality 3D consistency through a 3D Adapter and surpassing score distillation in both speed and quality.
paper_ai_keywords: SDEdit ancestral sampling 3D Adapter 2D diffusion models 3D consistency visual quality inference time score distillation text/image-to-3D generation 3D-to-3D editing high-quality texture synthesis latent diffusion models text-to-3D initialization