ShowRoom3D: Text to High-Quality 3D Room Generation Using 3D Priors
We introduce ShowRoom3D, a three-stage approach for generating high-quality
3D room-scale scenes from texts. Previous methods using 2D diffusion priors to
optimize neural radiance fields for generating room-scale scenes have shown
unsatisfactory quality. This is primarily attributed to the limitations of 2D
priors lacking 3D awareness and constraints in the training methodology. In
this paper, we utilize a 3D diffusion prior, MVDiffusion, to optimize the 3D
room-scale scene. Our contributions are in two aspects. Firstly, we propose a
progressive view selection process to optimize NeRF. This involves dividing the
training process into three stages, gradually expanding the camera sampling
scope. Secondly, we propose the pose transformation method in the second stage.
It will ensure MVDiffusion provide the accurate view guidance. As a result,
ShowRoom3D enables the generation of rooms with improved structural integrity,
enhanced clarity from any view, reduced content repetition, and higher
consistency across different perspectives. Extensive experiments demonstrate
that our method, significantly outperforms state-of-the-art approaches by a
large margin in terms of user study.
658513e42189951844e7397b
Weijia Mao
admin_assigned
2023-12-22T12:05:28.131Z
63f320ee0be81bdc5d8ecb88
/avatars/9d08cff6ed23a51887c869947bc03228.svg
Mao Weijia
user
benzweijia
658513e42189951844e7397c
Yan-Pei Cao
admin_assigned
2023-12-22T12:05:38.257Z
638066faf022c8a5803f7eb8
/avatars/4cfd699c3f6c5461b12b7dc5e3fe183d.svg
Yanpei Cao
user
pookiefoof
658513e42189951844e7397d
Jia-Wei Liu
admin_assigned
2023-12-22T12:06:22.545Z
64230048a73327caad9d8241
/avatars/ac115f1a21c743db7c925f0e18451145.svg
Jiawei Liu
user
JiaweiLIU
658513e42189951844e7397e
Zhongcong Xu
admin_assigned
2023-12-22T12:06:29.413Z
631c2bed7f7a1b6cb9f6b114
/avatars/97d2c0b6123691ea27157ebf8da59b45.svg
Zhongcong Xu
user
zcxu-eric
658513e42189951844e7397f
Mike Zheng Shou
admin_assigned
2023-12-22T12:06:41.146Z
63a55320ce5763e06f78519c
https://cdn-avatars.huggingface.co/v1/production/uploads/1671779060549-noauth.jpeg
Mike Shou
user
mikeshou
ShowRoom3D: Text to High-Quality 3D Room Generation Using 3D Priors
We introduce ShowRoom3D, a three-stage approach for generating high-quality
3D room-scale scenes from texts. Previous methods using 2D diffusion priors to
optimize neural radiance fields for generating room-scale scenes have shown
unsatisfactory quality. This is primarily attributed to the limitations of 2D
priors lacking 3D awareness and constraints in the training methodology. In
this paper, we utilize a 3D diffusion prior, MVDiffusion, to optimize the 3D
room-scale scene. Our contributions are in two aspects. Firstly, we propose a
progressive view selection process to optimize NeRF. This involves dividing the
training process into three stages, gradually expanding the camera sampling
scope. Secondly, we propose the pose transformation method in the second stage.
It will ensure MVDiffusion provide the accurate view guidance. As a result,
ShowRoom3D enables the generation of rooms with improved structural integrity,
enhanced clarity from any view, reduced content repetition, and higher
consistency across different perspectives. Extensive experiments demonstrate
that our method, significantly outperforms state-of-the-art approaches by a
large margin in terms of user study.
ShowRoom3D generates high-quality 3D room-scale scenes from texts using a 3D diffusion prior and a progressive view selection process, improving structural integrity and clarity across different perspectives.
MVDiffusion NeRF 3D diffusion prior progressive view selection pose transformation 3D room-scale scenes