title: HoloDreamer: Holistic 3D Panoramic World Generation from Text
    Descriptions
summary: 3D scene generation is in high demand across various domains, including
  virtual reality, gaming, and the film industry. Owing to the powerful
  generative capabilities of text-to-image diffusion models that provide reliable
  priors, the creation of 3D scenes using only text prompts has become viable,
  thereby significantly advancing researches in text-driven 3D scene generation.
  In order to obtain multiple-view supervision from 2D diffusion models,
  prevailing methods typically employ the diffusion model to generate an initial
  local image, followed by iteratively outpainting the local image using
  diffusion models to gradually generate scenes. Nevertheless, these
  outpainting-based approaches prone to produce global inconsistent scene
  generation results without high degree of completeness, restricting their
  broader applications. To tackle these problems, we introduce HoloDreamer, a
  framework that first generates high-definition panorama as a holistic
  initialization of the full 3D scene, then leverage 3D Gaussian Splatting
  (3D-GS) to quickly reconstruct the 3D scene, thereby facilitating the creation
  of view-consistent and fully enclosed 3D scenes. Specifically, we propose
  Stylized Equirectangular Panorama Generation, a pipeline that combines multiple
  diffusion models to enable stylized and detailed equirectangular panorama
  generation from complex text prompts. Subsequently, Enhanced Two-Stage Panorama
  Reconstruction is introduced, conducting a two-stage optimization of 3D-GS to
  inpaint the missing region and enhance the integrity of the scene.
  Comprehensive experiments demonstrated that our method outperforms prior works
  in terms of overall visual consistency and harmony as well as reconstruction
  quality and rendering robustness when generating fully enclosed scenes.
paper_authors._id: 669f3c081dc50f21bd2a27d5
paper_authors.name: Haiyang Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-07T08:11:07.717Z
paper_authors._id: 66eeda3676a8038cb448f11d
paper_authors.avatarUrl: /avatars/8d6e61f4c9354c6720ccaa7be0fe1d9f.svg
paper_authors.fullname: Haiyang Zhou
paper_authors.type: user
paper_authors.user: Marblueocean
paper_authors._id: 669f3c081dc50f21bd2a27d6
paper_authors.name: Xinhua Cheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-23T08:45:54.039Z
paper_authors._id: 645b4026f9d4ec91fdd516ce
paper_authors.avatarUrl: /avatars/fb6556ab973764af5c44980f4e8fcc71.svg
paper_authors.fullname: Xinhua Cheng
paper_authors.type: user
paper_authors.user: cxh0519
paper_authors._id: 669f3c081dc50f21bd2a27d7
paper_authors.name: Wangbo Yu
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-08-29T06:21:22.886Z
paper_authors._id: 63f095be6309c84d5f48848a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63f095be6309c84d5f48848a/pL2CKi-r-0mMfIGhYSAsm.jpeg
paper_authors.fullname: Wangbo Yu
paper_authors.type: user
paper_authors.user: Drexubery
paper_authors._id: 669f3c081dc50f21bd2a27d8
paper_authors.name: Yonghong Tian
paper_authors._id: 669f3c081dc50f21bd2a27d9
paper_authors.name: Li Yuan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-04T07:11:46.501Z
paper_authors._id: 66135a5e50350afe76beebce
paper_authors.avatarUrl: /avatars/370a4b83949355feb050c2cb0425c264.svg
paper_authors.fullname: yl2488
paper_authors.type: user
paper_authors.user: yl2488
paper_title: HoloDreamer: Holistic 3D Panoramic World Generation from Text
    Descriptions
paper_summary: 3D scene generation is in high demand across various domains, including
  virtual reality, gaming, and the film industry. Owing to the powerful
  generative capabilities of text-to-image diffusion models that provide reliable
  priors, the creation of 3D scenes using only text prompts has become viable,
  thereby significantly advancing researches in text-driven 3D scene generation.
  In order to obtain multiple-view supervision from 2D diffusion models,
  prevailing methods typically employ the diffusion model to generate an initial
  local image, followed by iteratively outpainting the local image using
  diffusion models to gradually generate scenes. Nevertheless, these
  outpainting-based approaches prone to produce global inconsistent scene
  generation results without high degree of completeness, restricting their
  broader applications. To tackle these problems, we introduce HoloDreamer, a
  framework that first generates high-definition panorama as a holistic
  initialization of the full 3D scene, then leverage 3D Gaussian Splatting
  (3D-GS) to quickly reconstruct the 3D scene, thereby facilitating the creation
  of view-consistent and fully enclosed 3D scenes. Specifically, we propose
  Stylized Equirectangular Panorama Generation, a pipeline that combines multiple
  diffusion models to enable stylized and detailed equirectangular panorama
  generation from complex text prompts. Subsequently, Enhanced Two-Stage Panorama
  Reconstruction is introduced, conducting a two-stage optimization of 3D-GS to
  inpaint the missing region and enhance the integrity of the scene.
  Comprehensive experiments demonstrated that our method outperforms prior works
  in terms of overall visual consistency and harmony as well as reconstruction
  quality and rendering robustness when generating fully enclosed scenes.
paper_ai_summary: HoloDreamer uses a stylized panorama generation pipeline and 3D Gaussian Splatting to create view-consistent and fully enclosed 3D scenes from text prompts, outperforming existing methods in visual consistency and reconstruction quality.
paper_ai_keywords: text-to-image diffusion models 3D Gaussian Splatting equirectangular panorama generation panorama reconstruction two-stage optimization