GaussianDreamer: Fast Generation from Text to 3D Gaussian Splatting with
  Point Cloud Priors
In recent times, the generation of 3D assets from text prompts has shown
impressive results. Both 2D and 3D diffusion models can generate decent 3D
objects based on prompts. 3D diffusion models have good 3D consistency, but
their quality and generalization are limited as trainable 3D data is expensive
and hard to obtain. 2D diffusion models enjoy strong abilities of
generalization and fine generation, but the 3D consistency is hard to
guarantee. This paper attempts to bridge the power from the two types of
diffusion models via the recent explicit and efficient 3D Gaussian splatting
representation. A fast 3D generation framework, named as \name, is proposed,
where the 3D diffusion model provides point cloud priors for initialization and
the 2D diffusion model enriches the geometry and appearance. Operations of
noisy point growing and color perturbation are introduced to enhance the
initialized Gaussians. Our \name can generate a high-quality 3D instance within
25 minutes on one GPU, much faster than previous methods, while the generated
instances can be directly rendered in real time. Demos and code are available
at https://taoranyi.com/gaussiandreamer/.
6529558d4d0eef4a2bd41b3a
Taoran Yi
claimed_verified
2023-10-16T08:25:30.621Z
64e5ee4b93d04e3439f4e988
https://cdn-avatars.huggingface.co/v1/production/uploads/64e5ee4b93d04e3439f4e988/g-THJvunoSPUyQ2v_fChU.jpeg
taoranyi
user
thewhole
6529558d4d0eef4a2bd41b3b
Jiemin Fang
claimed_verified
2023-10-16T08:40:03.502Z
652cf49cc6857682d3168f5a
/avatars/9ccffa04832429aad37320a301ea8e36.svg
Jiemin Fang
user
JieminFang
6529558d4d0eef4a2bd41b3c
Guanjun Wu
claimed_verified
2023-10-16T07:22:14.838Z
65297efc802e3d1a4f7e23e8
/avatars/09dc58dbbce5684e9edcb813bf078cc9.svg
Guanjun Wu
user
SnowGeraltOfRivia
6529558d4d0eef4a2bd41b3d
Lingxi Xie
6529558d4d0eef4a2bd41b3e
Xiaopeng Zhang
admin_assigned
2023-10-13T15:57:52.890Z
640bea9cb4f73086cddbe0a6
/avatars/ae460dd971de1d04660b266b22f81f9d.svg
Xiaopeng Zhang
user
xpzhang
6529558d4d0eef4a2bd41b3f
Wenyu Liu
6529558d4d0eef4a2bd41b40
Qi Tian
6529558d4d0eef4a2bd41b41
Xinggang Wang
admin_assigned
2023-10-13T16:03:02.428Z
62600de6d47e3dbae32ce1ce
/avatars/a536417cfec6e10ac415091bd1829426.svg
Xinggang Wang
user
xinggangw
GaussianDreamer: Fast Generation from Text to 3D Gaussian Splatting with
  Point Cloud Priors
In recent times, the generation of 3D assets from text prompts has shown
impressive results. Both 2D and 3D diffusion models can generate decent 3D
objects based on prompts. 3D diffusion models have good 3D consistency, but
their quality and generalization are limited as trainable 3D data is expensive
and hard to obtain. 2D diffusion models enjoy strong abilities of
generalization and fine generation, but the 3D consistency is hard to
guarantee. This paper attempts to bridge the power from the two types of
diffusion models via the recent explicit and efficient 3D Gaussian splatting
representation. A fast 3D generation framework, named as \name, is proposed,
where the 3D diffusion model provides point cloud priors for initialization and
the 2D diffusion model enriches the geometry and appearance. Operations of
noisy point growing and color perturbation are introduced to enhance the
initialized Gaussians. Our \name can generate a high-quality 3D instance within
25 minutes on one GPU, much faster than previous methods, while the generated
instances can be directly rendered in real time. Demos and code are available
at https://taoranyi.com/gaussiandreamer/.
A framework combining 2D and 3D diffusion models using 3D Gaussian splatting efficiently generates high-quality 3D assets from text prompts with real-time rendering.
3D diffusion models 2D diffusion models 3D Gaussian splatting noisy point growing color perturbation real-time rendering