HoloDreamer: Holistic 3D Panoramic World Generation from Text
  Descriptions
3D scene generation is in high demand across various domains, including
virtual reality, gaming, and the film industry. Owing to the powerful
generative capabilities of text-to-image diffusion models that provide reliable
priors, the creation of 3D scenes using only text prompts has become viable,
thereby significantly advancing researches in text-driven 3D scene generation.
In order to obtain multiple-view supervision from 2D diffusion models,
prevailing methods typically employ the diffusion model to generate an initial
local image, followed by iteratively outpainting the local image using
diffusion models to gradually generate scenes. Nevertheless, these
outpainting-based approaches prone to produce global inconsistent scene
generation results without high degree of completeness, restricting their
broader applications. To tackle these problems, we introduce HoloDreamer, a
framework that first generates high-definition panorama as a holistic
initialization of the full 3D scene, then leverage 3D Gaussian Splatting
(3D-GS) to quickly reconstruct the 3D scene, thereby facilitating the creation
of view-consistent and fully enclosed 3D scenes. Specifically, we propose
Stylized Equirectangular Panorama Generation, a pipeline that combines multiple
diffusion models to enable stylized and detailed equirectangular panorama
generation from complex text prompts. Subsequently, Enhanced Two-Stage Panorama
Reconstruction is introduced, conducting a two-stage optimization of 3D-GS to
inpaint the missing region and enhance the integrity of the scene.
Comprehensive experiments demonstrated that our method outperforms prior works
in terms of overall visual consistency and harmony as well as reconstruction
quality and rendering robustness when generating fully enclosed scenes.
669f3c081dc50f21bd2a27d5
Haiyang Zhou
claimed_verified
2025-05-07T08:11:07.717Z
66eeda3676a8038cb448f11d
/avatars/8d6e61f4c9354c6720ccaa7be0fe1d9f.svg
Haiyang Zhou
user
Marblueocean
669f3c081dc50f21bd2a27d6
Xinhua Cheng
admin_assigned
2024-07-23T08:45:54.039Z
645b4026f9d4ec91fdd516ce
/avatars/fb6556ab973764af5c44980f4e8fcc71.svg
Xinhua Cheng
user
cxh0519
669f3c081dc50f21bd2a27d7
Wangbo Yu
extracted_confirmed
2024-08-29T06:21:22.886Z
63f095be6309c84d5f48848a
https://cdn-avatars.huggingface.co/v1/production/uploads/63f095be6309c84d5f48848a/pL2CKi-r-0mMfIGhYSAsm.jpeg
Wangbo Yu
user
Drexubery
669f3c081dc50f21bd2a27d8
Yonghong Tian
669f3c081dc50f21bd2a27d9
Li Yuan
claimed_verified
2025-04-04T07:11:46.501Z
66135a5e50350afe76beebce
/avatars/370a4b83949355feb050c2cb0425c264.svg
yl2488
user
yl2488
HoloDreamer: Holistic 3D Panoramic World Generation from Text
  Descriptions
3D scene generation is in high demand across various domains, including
virtual reality, gaming, and the film industry. Owing to the powerful
generative capabilities of text-to-image diffusion models that provide reliable
priors, the creation of 3D scenes using only text prompts has become viable,
thereby significantly advancing researches in text-driven 3D scene generation.
In order to obtain multiple-view supervision from 2D diffusion models,
prevailing methods typically employ the diffusion model to generate an initial
local image, followed by iteratively outpainting the local image using
diffusion models to gradually generate scenes. Nevertheless, these
outpainting-based approaches prone to produce global inconsistent scene
generation results without high degree of completeness, restricting their
broader applications. To tackle these problems, we introduce HoloDreamer, a
framework that first generates high-definition panorama as a holistic
initialization of the full 3D scene, then leverage 3D Gaussian Splatting
(3D-GS) to quickly reconstruct the 3D scene, thereby facilitating the creation
of view-consistent and fully enclosed 3D scenes. Specifically, we propose
Stylized Equirectangular Panorama Generation, a pipeline that combines multiple
diffusion models to enable stylized and detailed equirectangular panorama
generation from complex text prompts. Subsequently, Enhanced Two-Stage Panorama
Reconstruction is introduced, conducting a two-stage optimization of 3D-GS to
inpaint the missing region and enhance the integrity of the scene.
Comprehensive experiments demonstrated that our method outperforms prior works
in terms of overall visual consistency and harmony as well as reconstruction
quality and rendering robustness when generating fully enclosed scenes.
HoloDreamer uses a stylized panorama generation pipeline and 3D Gaussian Splatting to create view-consistent and fully enclosed 3D scenes from text prompts, outperforming existing methods in visual consistency and reconstruction quality.
text-to-image diffusion models 3D Gaussian Splatting equirectangular panorama generation panorama reconstruction two-stage optimization