UniDream: Unifying Diffusion Priors for Relightable Text-to-3D
  Generation
Recent advancements in text-to-3D generation technology have significantly
advanced the conversion of textual descriptions into imaginative
well-geometrical and finely textured 3D objects. Despite these developments, a
prevalent limitation arises from the use of RGB data in diffusion or
reconstruction models, which often results in models with inherent lighting and
shadows effects that detract from their realism, thereby limiting their
usability in applications that demand accurate relighting capabilities. To
bridge this gap, we present UniDream, a text-to-3D generation framework by
incorporating unified diffusion priors. Our approach consists of three main
components: (1) a dual-phase training process to get albedo-normal aligned
multi-view diffusion and reconstruction models, (2) a progressive generation
procedure for geometry and albedo-textures based on Score Distillation Sample
(SDS) using the trained reconstruction and diffusion models, and (3) an
innovative application of SDS for finalizing PBR generation while keeping a
fixed albedo based on Stable Diffusion model. Extensive evaluations demonstrate
that UniDream surpasses existing methods in generating 3D objects with clearer
albedo textures, smoother surfaces, enhanced realism, and superior relighting
capabilities.
657bd69637d20b27ef5c9680
Zexiang Liu
657bd69637d20b27ef5c9681
Yangguang Li
admin_assigned
2023-12-15T13:10:58.547Z
644990cbdf4e6cb7eaee0559
/avatars/dbe8d4679a3b29cbb1cb7b3da13ed61f.svg
Yangguang
user
liyangguang
657bd69637d20b27ef5c9682
Youtian Lin
admin_assigned
2023-12-15T13:11:06.020Z
645a24779f06c5897254d14b
/avatars/dd0a635674025dcc9a94ee0f4c952083.svg
Youtian Lin
user
LoYoT
657bd69637d20b27ef5c9683
Xin Yu
claimed_verified
2025-03-12T15:26:10.990Z
62d1a71ac40bde8b986c3748
/avatars/6dbfba55563c71ed0db0d59c2629d525.svg
Xin Yu
user
Andyx
657bd69637d20b27ef5c9684
Sida Peng
admin_assigned
2023-12-15T13:11:22.563Z
62986ca2b58e71e2ac9b8f01
/avatars/83944db5f3dbb6f47c47c46fb2cb2849.svg
Sida Peng
user
pengsida
657bd69637d20b27ef5c9685
Yan-Pei Cao
admin_assigned
2023-12-15T13:12:25.180Z
638066faf022c8a5803f7eb8
/avatars/4cfd699c3f6c5461b12b7dc5e3fe183d.svg
Yanpei Cao
user
pookiefoof
657bd69637d20b27ef5c9686
Xiaojuan Qi
claimed_verified
2025-07-15T19:16:08.899Z
6875266f9cd3191dfddc7071
/avatars/64c581910833b111e9a7bae5b8740229.svg
xiaojuan qi
user
xjqi
657bd69637d20b27ef5c9687
Xiaoshui Huang
657bd69637d20b27ef5c9688
Ding Liang
admin_assigned
2023-12-15T13:12:52.906Z
6476b01cbb7fdd7f425dcefb
/avatars/65efe7067f47a68c204a1ab5a772b939.svg
dingliang
user
dingliang01
657bd69637d20b27ef5c9689
Wanli Ouyang
UniDream: Unifying Diffusion Priors for Relightable Text-to-3D
  Generation
Recent advancements in text-to-3D generation technology have significantly
advanced the conversion of textual descriptions into imaginative
well-geometrical and finely textured 3D objects. Despite these developments, a
prevalent limitation arises from the use of RGB data in diffusion or
reconstruction models, which often results in models with inherent lighting and
shadows effects that detract from their realism, thereby limiting their
usability in applications that demand accurate relighting capabilities. To
bridge this gap, we present UniDream, a text-to-3D generation framework by
incorporating unified diffusion priors. Our approach consists of three main
components: (1) a dual-phase training process to get albedo-normal aligned
multi-view diffusion and reconstruction models, (2) a progressive generation
procedure for geometry and albedo-textures based on Score Distillation Sample
(SDS) using the trained reconstruction and diffusion models, and (3) an
innovative application of SDS for finalizing PBR generation while keeping a
fixed albedo based on Stable Diffusion model. Extensive evaluations demonstrate
that UniDream surpasses existing methods in generating 3D objects with clearer
albedo textures, smoother surfaces, enhanced realism, and superior relighting
capabilities.
UniDream is a text-to-3D generation framework using unified diffusion priors to produce realistic 3D objects with accurate relighting capabilities and improved albedo textures.
diffusion models reconstruction models dual-phase training progressive generation Score Distillation Sample (SDS) Stable Diffusion model albedo-textures relighting capabilities