title: GaussianDreamer: Fast Generation from Text to 3D Gaussian Splatting with
    Point Cloud Priors
summary: In recent times, the generation of 3D assets from text prompts has shown
  impressive results. Both 2D and 3D diffusion models can generate decent 3D
  objects based on prompts. 3D diffusion models have good 3D consistency, but
  their quality and generalization are limited as trainable 3D data is expensive
  and hard to obtain. 2D diffusion models enjoy strong abilities of
  generalization and fine generation, but the 3D consistency is hard to
  guarantee. This paper attempts to bridge the power from the two types of
  diffusion models via the recent explicit and efficient 3D Gaussian splatting
  representation. A fast 3D generation framework, named as \name, is proposed,
  where the 3D diffusion model provides point cloud priors for initialization and
  the 2D diffusion model enriches the geometry and appearance. Operations of
  noisy point growing and color perturbation are introduced to enhance the
  initialized Gaussians. Our \name can generate a high-quality 3D instance within
  25 minutes on one GPU, much faster than previous methods, while the generated
  instances can be directly rendered in real time. Demos and code are available
  at https://taoranyi.com/gaussiandreamer/.
paper_authors._id: 6529558d4d0eef4a2bd41b3a
paper_authors.name: Taoran Yi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-16T08:25:30.621Z
paper_authors._id: 64e5ee4b93d04e3439f4e988
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64e5ee4b93d04e3439f4e988/g-THJvunoSPUyQ2v_fChU.jpeg
paper_authors.fullname: taoranyi
paper_authors.type: user
paper_authors.user: thewhole
paper_authors._id: 6529558d4d0eef4a2bd41b3b
paper_authors.name: Jiemin Fang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-16T08:40:03.502Z
paper_authors._id: 652cf49cc6857682d3168f5a
paper_authors.avatarUrl: /avatars/9ccffa04832429aad37320a301ea8e36.svg
paper_authors.fullname: Jiemin Fang
paper_authors.type: user
paper_authors.user: JieminFang
paper_authors._id: 6529558d4d0eef4a2bd41b3c
paper_authors.name: Guanjun Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-16T07:22:14.838Z
paper_authors._id: 65297efc802e3d1a4f7e23e8
paper_authors.avatarUrl: /avatars/09dc58dbbce5684e9edcb813bf078cc9.svg
paper_authors.fullname: Guanjun Wu
paper_authors.type: user
paper_authors.user: SnowGeraltOfRivia
paper_authors._id: 6529558d4d0eef4a2bd41b3d
paper_authors.name: Lingxi Xie
paper_authors._id: 6529558d4d0eef4a2bd41b3e
paper_authors.name: Xiaopeng Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-13T15:57:52.890Z
paper_authors._id: 640bea9cb4f73086cddbe0a6
paper_authors.avatarUrl: /avatars/ae460dd971de1d04660b266b22f81f9d.svg
paper_authors.fullname: Xiaopeng Zhang
paper_authors.type: user
paper_authors.user: xpzhang
paper_authors._id: 6529558d4d0eef4a2bd41b3f
paper_authors.name: Wenyu Liu
paper_authors._id: 6529558d4d0eef4a2bd41b40
paper_authors.name: Qi Tian
paper_authors._id: 6529558d4d0eef4a2bd41b41
paper_authors.name: Xinggang Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-13T16:03:02.428Z
paper_authors._id: 62600de6d47e3dbae32ce1ce
paper_authors.avatarUrl: /avatars/a536417cfec6e10ac415091bd1829426.svg
paper_authors.fullname: Xinggang Wang
paper_authors.type: user
paper_authors.user: xinggangw
paper_title: GaussianDreamer: Fast Generation from Text to 3D Gaussian Splatting with
    Point Cloud Priors
paper_summary: In recent times, the generation of 3D assets from text prompts has shown
  impressive results. Both 2D and 3D diffusion models can generate decent 3D
  objects based on prompts. 3D diffusion models have good 3D consistency, but
  their quality and generalization are limited as trainable 3D data is expensive
  and hard to obtain. 2D diffusion models enjoy strong abilities of
  generalization and fine generation, but the 3D consistency is hard to
  guarantee. This paper attempts to bridge the power from the two types of
  diffusion models via the recent explicit and efficient 3D Gaussian splatting
  representation. A fast 3D generation framework, named as \name, is proposed,
  where the 3D diffusion model provides point cloud priors for initialization and
  the 2D diffusion model enriches the geometry and appearance. Operations of
  noisy point growing and color perturbation are introduced to enhance the
  initialized Gaussians. Our \name can generate a high-quality 3D instance within
  25 minutes on one GPU, much faster than previous methods, while the generated
  instances can be directly rendered in real time. Demos and code are available
  at https://taoranyi.com/gaussiandreamer/.
paper_ai_summary: A framework combining 2D and 3D diffusion models using 3D Gaussian splatting efficiently generates high-quality 3D assets from text prompts with real-time rendering.
paper_ai_keywords: 3D diffusion models 2D diffusion models 3D Gaussian splatting noisy point growing color perturbation real-time rendering