DreamScene: 3D Gaussian-based End-to-end Text-to-3D Scene Generation
Generating 3D scenes from natural language holds great promise for
applications in gaming, film, and design. However, existing methods struggle
with automation, 3D consistency, and fine-grained control. We present
DreamScene, an end-to-end framework for high-quality and editable 3D scene
generation from text or dialogue. DreamScene begins with a scene planning
module, where a GPT-4 agent infers object semantics and spatial constraints to
construct a hybrid graph. A graph-based placement algorithm then produces a
structured, collision-free layout. Based on this layout, Formation Pattern
Sampling (FPS) generates object geometry using multi-timestep sampling and
reconstructive optimization, enabling fast and realistic synthesis. To ensure
global consistent, DreamScene employs a progressive camera sampling strategy
tailored to both indoor and outdoor settings. Finally, the system supports
fine-grained scene editing, including object movement, appearance changes, and
4D dynamic motion. Experiments demonstrate that DreamScene surpasses prior
methods in quality, consistency, and flexibility, offering a practical solution
for open-domain 3D content creation. Code and demos are available at
https://jahnsonblack.github.io/DreamScene-Full/.
688b7d308c434640078cc2a5
Haoran Li
688b7d308c434640078cc2a6
Yuli Tian
688b7d308c434640078cc2a7
Kun Lan
688b7d308c434640078cc2a8
Yong Liao
688b7d308c434640078cc2a9
Lin Wang
688b7d308c434640078cc2aa
Pan Hui
688b7d308c434640078cc2ab
Peng Yuan Zhou
DreamScene: 3D Gaussian-based End-to-end Text-to-3D Scene Generation
Generating 3D scenes from natural language holds great promise for
applications in gaming, film, and design. However, existing methods struggle
with automation, 3D consistency, and fine-grained control. We present
DreamScene, an end-to-end framework for high-quality and editable 3D scene
generation from text or dialogue. DreamScene begins with a scene planning
module, where a GPT-4 agent infers object semantics and spatial constraints to
construct a hybrid graph. A graph-based placement algorithm then produces a
structured, collision-free layout. Based on this layout, Formation Pattern
Sampling (FPS) generates object geometry using multi-timestep sampling and
reconstructive optimization, enabling fast and realistic synthesis. To ensure
global consistent, DreamScene employs a progressive camera sampling strategy
tailored to both indoor and outdoor settings. Finally, the system supports
fine-grained scene editing, including object movement, appearance changes, and
4D dynamic motion. Experiments demonstrate that DreamScene surpasses prior
methods in quality, consistency, and flexibility, offering a practical solution
for open-domain 3D content creation. Code and demos are available at
https://jahnsonblack.github.io/DreamScene-Full/.
DreamScene is an end-to-end framework that generates high-quality, editable 3D scenes from text or dialogue, ensuring automation, 3D consistency, and fine-grained control through a combination of scene planning, graph-based placement, formation pattern sampling, and progressive camera sampling.
GPT-4 hybrid graph graph-based placement algorithm Formation Pattern Sampling multi-timestep sampling reconstructive optimization progressive camera sampling 3D scene generation scene editing 4D dynamic motion