DreamCube: 3D Panorama Generation via Multi-plane Synchronization
3D panorama synthesis is a promising yet challenging task that demands
high-quality and diverse visual appearance and geometry of the generated
omnidirectional content. Existing methods leverage rich image priors from
pre-trained 2D foundation models to circumvent the scarcity of 3D panoramic
data, but the incompatibility between 3D panoramas and 2D single views limits
their effectiveness. In this work, we demonstrate that by applying multi-plane
synchronization to the operators from 2D foundation models, their capabilities
can be seamlessly extended to the omnidirectional domain. Based on this design,
we further introduce DreamCube, a multi-plane RGB-D diffusion model for 3D
panorama generation, which maximizes the reuse of 2D foundation model priors to
achieve diverse appearances and accurate geometry while maintaining multi-view
consistency. Extensive experiments demonstrate the effectiveness of our
approach in panoramic image generation, panoramic depth estimation, and 3D
scene generation.
6858c5b5c0c8e29df8ea3c95
Yukun Huang
claimed_verified
2025-06-24T08:10:01.006Z
638ee900ee7e45e0474a5712
https://cdn-avatars.huggingface.co/v1/production/uploads/638ee900ee7e45e0474a5712/KLli_eCbWwffKR7oLDmV3.jpeg
Yukun Huang
user
KevinHuang
6858c5b5c0c8e29df8ea3c96
Yanning Zhou
6858c5b5c0c8e29df8ea3c97
Jianan Wang
6858c5b5c0c8e29df8ea3c98
Kaiyi Huang
6858c5b5c0c8e29df8ea3c99
Xihui Liu
DreamCube: 3D Panorama Generation via Multi-plane Synchronization
3D panorama synthesis is a promising yet challenging task that demands
high-quality and diverse visual appearance and geometry of the generated
omnidirectional content. Existing methods leverage rich image priors from
pre-trained 2D foundation models to circumvent the scarcity of 3D panoramic
data, but the incompatibility between 3D panoramas and 2D single views limits
their effectiveness. In this work, we demonstrate that by applying multi-plane
synchronization to the operators from 2D foundation models, their capabilities
can be seamlessly extended to the omnidirectional domain. Based on this design,
we further introduce DreamCube, a multi-plane RGB-D diffusion model for 3D
panorama generation, which maximizes the reuse of 2D foundation model priors to
achieve diverse appearances and accurate geometry while maintaining multi-view
consistency. Extensive experiments demonstrate the effectiveness of our
approach in panoramic image generation, panoramic depth estimation, and 3D
scene generation.
Multi-plane synchronization extends 2D foundation models to 3D panorama generation, introducing DreamCube to achieve diverse appearances and accurate geometry.
multi-plane synchronization 2D foundation models DreamCube RGB-D diffusion model panoramic image generation panoramic depth estimation 3D scene generation