title: DiffusionGAN3D: Boosting Text-guided 3D Generation and Domain Adaption
    by Combining 3D GANs and Diffusion Priors
summary: Text-guided domain adaption and generation of 3D-aware portraits find many
  applications in various fields. However, due to the lack of training data and
  the challenges in handling the high variety of geometry and appearance, the
  existing methods for these tasks suffer from issues like inflexibility,
  instability, and low fidelity. In this paper, we propose a novel framework
  DiffusionGAN3D, which boosts text-guided 3D domain adaption and generation by
  combining 3D GANs and diffusion priors. Specifically, we integrate the
  pre-trained 3D generative models (e.g., EG3D) and text-to-image diffusion
  models. The former provides a strong foundation for stable and high-quality
  avatar generation from text. And the diffusion models in turn offer powerful
  priors and guide the 3D generator finetuning with informative direction to
  achieve flexible and efficient text-guided domain adaption. To enhance the
  diversity in domain adaption and the generation capability in text-to-avatar,
  we introduce the relative distance loss and case-specific learnable triplane
  respectively. Besides, we design a progressive texture refinement module to
  improve the texture quality for both tasks above. Extensive experiments
  demonstrate that the proposed framework achieves excellent results in both
  domain adaption and text-to-avatar tasks, outperforming existing methods in
  terms of generation quality and efficiency. The project homepage is at
  https://younglbw.github.io/DiffusionGAN3D-homepage/.
paper_authors._id: 658e4279bea632dc942f5655
paper_authors.name: Biwen Lei
paper_authors._id: 658e4279bea632dc942f5656
paper_authors.name: Kai Yu
paper_authors._id: 658e4279bea632dc942f5657
paper_authors.name: Mengyang Feng
paper_authors._id: 658e4279bea632dc942f5658
paper_authors.name: Miaomiao Cui
paper_authors._id: 658e4279bea632dc942f5659
paper_authors.name: Xuansong Xie
paper_title: DiffusionGAN3D: Boosting Text-guided 3D Generation and Domain Adaption
    by Combining 3D GANs and Diffusion Priors
paper_summary: Text-guided domain adaption and generation of 3D-aware portraits find many
  applications in various fields. However, due to the lack of training data and
  the challenges in handling the high variety of geometry and appearance, the
  existing methods for these tasks suffer from issues like inflexibility,
  instability, and low fidelity. In this paper, we propose a novel framework
  DiffusionGAN3D, which boosts text-guided 3D domain adaption and generation by
  combining 3D GANs and diffusion priors. Specifically, we integrate the
  pre-trained 3D generative models (e.g., EG3D) and text-to-image diffusion
  models. The former provides a strong foundation for stable and high-quality
  avatar generation from text. And the diffusion models in turn offer powerful
  priors and guide the 3D generator finetuning with informative direction to
  achieve flexible and efficient text-guided domain adaption. To enhance the
  diversity in domain adaption and the generation capability in text-to-avatar,
  we introduce the relative distance loss and case-specific learnable triplane
  respectively. Besides, we design a progressive texture refinement module to
  improve the texture quality for both tasks above. Extensive experiments
  demonstrate that the proposed framework achieves excellent results in both
  domain adaption and text-to-avatar tasks, outperforming existing methods in
  terms of generation quality and efficiency. The project homepage is at
  https://younglbw.github.io/DiffusionGAN3D-homepage/.
paper_ai_keywords: DiffusionGAN3D 3D GANs diffusion priors EG3D text-to-image diffusion models relative distance loss learnable feature planes progressive texture refinement module domain adaptation text-to-avatar generation quality efficiency