DreamTime: An Improved Optimization Strategy for Text-to-3D Content
  Creation
Text-to-image diffusion models pre-trained on billions of image-text pairs
have recently enabled text-to-3D content creation by optimizing a randomly
initialized Neural Radiance Fields (NeRF) with score distillation. However, the
resultant 3D models exhibit two limitations: (a) quality concerns such as
saturated color and the Janus problem; (b) extremely low diversity comparing to
text-guided image synthesis. In this paper, we show that the conflict between
NeRF optimization process and uniform timestep sampling in score distillation
is the main reason for these limitations. To resolve this conflict, we propose
to prioritize timestep sampling with monotonically non-increasing functions,
which aligns NeRF optimization with the sampling process of diffusion model.
Extensive experiments show that our simple redesign significantly improves
text-to-3D content creation with higher quality and diversity.
6493c7d774db6cc30f41a594
Yukun Huang
claimed_verified
2023-06-23T07:31:55.127Z
638ee900ee7e45e0474a5712
https://cdn-avatars.huggingface.co/v1/production/uploads/638ee900ee7e45e0474a5712/KLli_eCbWwffKR7oLDmV3.jpeg
Yukun Huang
user
KevinHuang
6493c7d774db6cc30f41a595
Jianan Wang
6493c7d774db6cc30f41a596
Yukai Shi
6493c7d774db6cc30f41a597
Xianbiao Qi
6493c7d774db6cc30f41a598
Zheng-Jun Zha
6493c7d774db6cc30f41a599
Lei Zhang
DreamTime: An Improved Optimization Strategy for Text-to-3D Content
  Creation
Text-to-image diffusion models pre-trained on billions of image-text pairs
have recently enabled text-to-3D content creation by optimizing a randomly
initialized Neural Radiance Fields (NeRF) with score distillation. However, the
resultant 3D models exhibit two limitations: (a) quality concerns such as
saturated color and the Janus problem; (b) extremely low diversity comparing to
text-guided image synthesis. In this paper, we show that the conflict between
NeRF optimization process and uniform timestep sampling in score distillation
is the main reason for these limitations. To resolve this conflict, we propose
to prioritize timestep sampling with monotonically non-increasing functions,
which aligns NeRF optimization with the sampling process of diffusion model.
Extensive experiments show that our simple redesign significantly improves
text-to-3D content creation with higher quality and diversity.
Proposing a redesign in timestep sampling for text-to-3D diffusion models to enhance quality and diversity by aligning with NeRF optimization.
diffusion models text-to-3D Neural Radiance Fields (NeRF) score distillation timestep sampling monotonically non-increasing functions