title: UniDream: Unifying Diffusion Priors for Relightable Text-to-3D
    Generation
summary: Recent advancements in text-to-3D generation technology have significantly
  advanced the conversion of textual descriptions into imaginative
  well-geometrical and finely textured 3D objects. Despite these developments, a
  prevalent limitation arises from the use of RGB data in diffusion or
  reconstruction models, which often results in models with inherent lighting and
  shadows effects that detract from their realism, thereby limiting their
  usability in applications that demand accurate relighting capabilities. To
  bridge this gap, we present UniDream, a text-to-3D generation framework by
  incorporating unified diffusion priors. Our approach consists of three main
  components: (1) a dual-phase training process to get albedo-normal aligned
  multi-view diffusion and reconstruction models, (2) a progressive generation
  procedure for geometry and albedo-textures based on Score Distillation Sample
  (SDS) using the trained reconstruction and diffusion models, and (3) an
  innovative application of SDS for finalizing PBR generation while keeping a
  fixed albedo based on Stable Diffusion model. Extensive evaluations demonstrate
  that UniDream surpasses existing methods in generating 3D objects with clearer
  albedo textures, smoother surfaces, enhanced realism, and superior relighting
  capabilities.
paper_authors._id: 657bd69637d20b27ef5c9680
paper_authors.name: Zexiang Liu
paper_authors._id: 657bd69637d20b27ef5c9681
paper_authors.name: Yangguang Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-15T13:10:58.547Z
paper_authors._id: 644990cbdf4e6cb7eaee0559
paper_authors.avatarUrl: /avatars/dbe8d4679a3b29cbb1cb7b3da13ed61f.svg
paper_authors.fullname: Yangguang
paper_authors.type: user
paper_authors.user: liyangguang
paper_authors._id: 657bd69637d20b27ef5c9682
paper_authors.name: Youtian Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-15T13:11:06.020Z
paper_authors._id: 645a24779f06c5897254d14b
paper_authors.avatarUrl: /avatars/dd0a635674025dcc9a94ee0f4c952083.svg
paper_authors.fullname: Youtian Lin
paper_authors.type: user
paper_authors.user: LoYoT
paper_authors._id: 657bd69637d20b27ef5c9683
paper_authors.name: Xin Yu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-12T15:26:10.990Z
paper_authors._id: 62d1a71ac40bde8b986c3748
paper_authors.avatarUrl: /avatars/6dbfba55563c71ed0db0d59c2629d525.svg
paper_authors.fullname: Xin Yu
paper_authors.type: user
paper_authors.user: Andyx
paper_authors._id: 657bd69637d20b27ef5c9684
paper_authors.name: Sida Peng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-15T13:11:22.563Z
paper_authors._id: 62986ca2b58e71e2ac9b8f01
paper_authors.avatarUrl: /avatars/83944db5f3dbb6f47c47c46fb2cb2849.svg
paper_authors.fullname: Sida Peng
paper_authors.type: user
paper_authors.user: pengsida
paper_authors._id: 657bd69637d20b27ef5c9685
paper_authors.name: Yan-Pei Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-15T13:12:25.180Z
paper_authors._id: 638066faf022c8a5803f7eb8
paper_authors.avatarUrl: /avatars/4cfd699c3f6c5461b12b7dc5e3fe183d.svg
paper_authors.fullname: Yanpei Cao
paper_authors.type: user
paper_authors.user: pookiefoof
paper_authors._id: 657bd69637d20b27ef5c9686
paper_authors.name: Xiaojuan Qi
paper_authors._id: 657bd69637d20b27ef5c9687
paper_authors.name: Xiaoshui Huang
paper_authors._id: 657bd69637d20b27ef5c9688
paper_authors.name: Ding Liang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-15T13:12:52.906Z
paper_authors._id: 6476b01cbb7fdd7f425dcefb
paper_authors.avatarUrl: /avatars/65efe7067f47a68c204a1ab5a772b939.svg
paper_authors.fullname: dingliang
paper_authors.type: user
paper_authors.user: dingliang01
paper_authors._id: 657bd69637d20b27ef5c9689
paper_authors.name: Wanli Ouyang
paper_title: UniDream: Unifying Diffusion Priors for Relightable Text-to-3D
    Generation
paper_summary: Recent advancements in text-to-3D generation technology have significantly
  advanced the conversion of textual descriptions into imaginative
  well-geometrical and finely textured 3D objects. Despite these developments, a
  prevalent limitation arises from the use of RGB data in diffusion or
  reconstruction models, which often results in models with inherent lighting and
  shadows effects that detract from their realism, thereby limiting their
  usability in applications that demand accurate relighting capabilities. To
  bridge this gap, we present UniDream, a text-to-3D generation framework by
  incorporating unified diffusion priors. Our approach consists of three main
  components: (1) a dual-phase training process to get albedo-normal aligned
  multi-view diffusion and reconstruction models, (2) a progressive generation
  procedure for geometry and albedo-textures based on Score Distillation Sample
  (SDS) using the trained reconstruction and diffusion models, and (3) an
  innovative application of SDS for finalizing PBR generation while keeping a
  fixed albedo based on Stable Diffusion model. Extensive evaluations demonstrate
  that UniDream surpasses existing methods in generating 3D objects with clearer
  albedo textures, smoother surfaces, enhanced realism, and superior relighting
  capabilities.
paper_ai_summary: UniDream is a text-to-3D generation framework using unified diffusion priors to produce realistic 3D objects with accurate relighting capabilities and improved albedo textures.
paper_ai_keywords: diffusion models reconstruction models dual-phase training progressive generation Score Distillation Sample (SDS) Stable Diffusion model albedo-textures relighting capabilities