title: DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic
    Gaussian Splatting
summary: The increasing demand for virtual reality applications has highlighted the
  significance of crafting immersive 3D assets. We present a text-to-3D
  360^{circ} scene generation pipeline that facilitates the creation of
  comprehensive 360^{circ} scenes for in-the-wild environments in a matter of
  minutes. Our approach utilizes the generative power of a 2D diffusion model and
  prompt self-refinement to create a high-quality and globally coherent panoramic
  image. This image acts as a preliminary "flat" (2D) scene representation.
  Subsequently, it is lifted into 3D Gaussians, employing splatting techniques to
  enable real-time exploration. To produce consistent 3D geometry, our pipeline
  constructs a spatially coherent structure by aligning the 2D monocular depth
  into a globally optimized point cloud. This point cloud serves as the initial
  state for the centroids of 3D Gaussians. In order to address invisible issues
  inherent in single-view inputs, we impose semantic and geometric constraints on
  both synthesized and input camera views as regularizations. These guide the
  optimization of Gaussians, aiding in the reconstruction of unseen regions. In
  summary, our method offers a globally consistent 3D scene within a
  360^{circ} perspective, providing an enhanced immersive experience over
  existing techniques. Project website at: http://dreamscene360.github.io/
paper_authors._id: 661743070795b7cd10a2ade6
paper_authors.name: Shijie Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-11T06:42:05.581Z
paper_authors._id: 642a276516d4d8293c9a47e8
paper_authors.avatarUrl: /avatars/80e6db8bc2544f3486b11b57858a8692.svg
paper_authors.fullname: Shijie Zhou
paper_authors.type: user
paper_authors.user: shijiezhou
paper_authors._id: 661743070795b7cd10a2ade7
paper_authors.name: Zhiwen Fan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-18T13:24:16.216Z
paper_authors._id: 6526386e1c6a09292d8d0a22
paper_authors.avatarUrl: /avatars/471de830de2d775d35368678c1579f87.svg
paper_authors.fullname: fan
paper_authors.type: user
paper_authors.user: Fanzhiwen
paper_authors._id: 661743070795b7cd10a2ade8
paper_authors.name: Dejia Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-18T13:24:25.149Z
paper_authors._id: 62f687a0c58915315c4ff75d
paper_authors.avatarUrl: /avatars/b657180c7666735062782edd4f6a69c9.svg
paper_authors.fullname: Dejia Xu
paper_authors.type: user
paper_authors.user: ir1d
paper_authors._id: 661743070795b7cd10a2ade9
paper_authors.name: Haoran Chang
paper_authors._id: 661743070795b7cd10a2adea
paper_authors.name: Pradyumna Chari
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-18T13:24:42.880Z
paper_authors._id: 64e2ac52cf8c236f30176499
paper_authors.avatarUrl: /avatars/0438be7e7b502a2892444249b49b92fe.svg
paper_authors.fullname: Pradyumna Chari
paper_authors.type: user
paper_authors.user: pradyumnachari
paper_authors._id: 661743070795b7cd10a2adeb
paper_authors.name: Tejas Bharadwaj
paper_authors._id: 661743070795b7cd10a2adec
paper_authors.name: Suya You
paper_authors._id: 661743070795b7cd10a2aded
paper_authors.name: Zhangyang Wang
paper_authors._id: 661743070795b7cd10a2adee
paper_authors.name: Achuta Kadambi
paper_title: DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic
    Gaussian Splatting
paper_summary: The increasing demand for virtual reality applications has highlighted the
  significance of crafting immersive 3D assets. We present a text-to-3D
  360^{circ} scene generation pipeline that facilitates the creation of
  comprehensive 360^{circ} scenes for in-the-wild environments in a matter of
  minutes. Our approach utilizes the generative power of a 2D diffusion model and
  prompt self-refinement to create a high-quality and globally coherent panoramic
  image. This image acts as a preliminary "flat" (2D) scene representation.
  Subsequently, it is lifted into 3D Gaussians, employing splatting techniques to
  enable real-time exploration. To produce consistent 3D geometry, our pipeline
  constructs a spatially coherent structure by aligning the 2D monocular depth
  into a globally optimized point cloud. This point cloud serves as the initial
  state for the centroids of 3D Gaussians. In order to address invisible issues
  inherent in single-view inputs, we impose semantic and geometric constraints on
  both synthesized and input camera views as regularizations. These guide the
  optimization of Gaussians, aiding in the reconstruction of unseen regions. In
  summary, our method offers a globally consistent 3D scene within a
  360^{circ} perspective, providing an enhanced immersive experience over
  existing techniques. Project website at: http://dreamscene360.github.io/
paper_ai_summary: A text-to-3D 360° scene generation pipeline creates immersive environments using a 2D diffusion model and lifts flat panoramas into 3D Gaussians with semantic and geometric constraints.
paper_ai_keywords: diffusion model prompt self-refinement panoramic image 3D Gaussians splatting techniques 2D monocular depth point cloud semantic constraints geometric constraints