title: Dual3D: Efficient and Consistent Text-to-3D Generation with Dual-mode
    Multi-view Latent Diffusion
summary: We present Dual3D, a novel text-to-3D generation framework that generates
  high-quality 3D assets from texts in only 1 minute.The key component is a
  dual-mode multi-view latent diffusion model. Given the noisy multi-view
  latents, the 2D mode can efficiently denoise them with a single latent
  denoising network, while the 3D mode can generate a tri-plane neural surface
  for consistent rendering-based denoising. Most modules for both modes are tuned
  from a pre-trained text-to-image latent diffusion model to circumvent the
  expensive cost of training from scratch. To overcome the high rendering cost
  during inference, we propose the dual-mode toggling inference strategy to use
  only 1/10 denoising steps with 3D mode, successfully generating a 3D asset in
  just 10 seconds without sacrificing quality. The texture of the 3D asset can
  be further enhanced by our efficient texture refinement process in a short
  time. Extensive experiments demonstrate that our method delivers
  state-of-the-art performance while significantly reducing generation time. Our
  project page is available at https://dual3d.github.io
paper_authors._id: 6646b824b4e78ba0e1f73776
paper_authors.name: Xinyang Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-05-20T07:38:37.794Z
paper_authors._id: 629631565de6e0eb3292afed
paper_authors.avatarUrl: /avatars/0ae1080eebce0f747f650bfc292c46ca.svg
paper_authors.fullname: Xinyang Li
paper_authors.type: user
paper_authors.user: imlixinyang
paper_authors._id: 6646b824b4e78ba0e1f73777
paper_authors.name: Zhangyu Lai
paper_authors._id: 6646b824b4e78ba0e1f73778
paper_authors.name: Linning Xu
paper_authors._id: 6646b824b4e78ba0e1f73779
paper_authors.name: Jianfei Guo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-17T11:04:51.347Z
paper_authors._id: 651507a1fc2301d62d7f32c5
paper_authors.avatarUrl: /avatars/1f1066ef5770a591afe0a1172f13125d.svg
paper_authors.fullname: guojianfei0228
paper_authors.type: user
paper_authors.user: guojianfei
paper_authors._id: 6646b824b4e78ba0e1f7377a
paper_authors.name: Liujuan Cao
paper_authors._id: 6646b824b4e78ba0e1f7377b
paper_authors.name: Shengchuan Zhang
paper_authors._id: 6646b824b4e78ba0e1f7377c
paper_authors.name: Bo Dai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-17T11:04:23.420Z
paper_authors._id: 635f93577c05eb9f59966209
paper_authors.avatarUrl: /avatars/add48d7e3790a6b500d6c451ef8b0f75.svg
paper_authors.fullname: Intelligent Digital Creation
paper_authors.type: user
paper_authors.user: BoDai
paper_authors._id: 6646b824b4e78ba0e1f7377d
paper_authors.name: Rongrong Ji
paper_title: Dual3D: Efficient and Consistent Text-to-3D Generation with Dual-mode
    Multi-view Latent Diffusion
paper_summary: We present Dual3D, a novel text-to-3D generation framework that generates
  high-quality 3D assets from texts in only 1 minute.The key component is a
  dual-mode multi-view latent diffusion model. Given the noisy multi-view
  latents, the 2D mode can efficiently denoise them with a single latent
  denoising network, while the 3D mode can generate a tri-plane neural surface
  for consistent rendering-based denoising. Most modules for both modes are tuned
  from a pre-trained text-to-image latent diffusion model to circumvent the
  expensive cost of training from scratch. To overcome the high rendering cost
  during inference, we propose the dual-mode toggling inference strategy to use
  only 1/10 denoising steps with 3D mode, successfully generating a 3D asset in
  just 10 seconds without sacrificing quality. The texture of the 3D asset can
  be further enhanced by our efficient texture refinement process in a short
  time. Extensive experiments demonstrate that our method delivers
  state-of-the-art performance while significantly reducing generation time. Our
  project page is available at https://dual3d.github.io
paper_ai_summary: Dual3D is a text-to-3D generation framework that combines dual-mode latent diffusion models with neural surfaces and pre-trained components for rapid and high-quality 3D asset creation.
paper_ai_keywords: dual-mode multi-view latent diffusion model latent denoising network tri-plane neural surface rendering-based denoising dual-mode toggling inference strategy