title: LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generation
summary: 3D immersive scene generation is a challenging yet critical task in computer
  vision and graphics. A desired virtual 3D scene should 1) exhibit
  omnidirectional view consistency, and 2) allow for free exploration in complex
  scene hierarchies. Existing methods either rely on successive scene expansion
  via inpainting or employ panorama representation to represent large FOV scene
  environments. However, the generated scene suffers from semantic drift during
  expansion and is unable to handle occlusion among scene hierarchies. To tackle
  these challenges, we introduce LayerPano3D, a novel framework for full-view,
  explorable panoramic 3D scene generation from a single text prompt. Our key
  insight is to decompose a reference 2D panorama into multiple layers at
  different depth levels, where each layer reveals the unseen space from the
  reference views via diffusion prior. LayerPano3D comprises multiple dedicated
  designs: 1) we introduce a novel text-guided anchor view synthesis pipeline for
  high-quality, consistent panorama generation. 2) We pioneer the Layered 3D
  Panorama as underlying representation to manage complex scene hierarchies and
  lift it into 3D Gaussians to splat detailed 360-degree omnidirectional scenes
  with unconstrained viewing paths. Extensive experiments demonstrate that our
  framework generates state-of-the-art 3D panoramic scene in both full view
  consistency and immersive exploratory experience. We believe that LayerPano3D
  holds promise for advancing 3D panoramic scene creation with numerous
  applications.
paper_authors._id: 66cbfc93888e99a28d4724c4
paper_authors.name: Shuai Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-06T07:43:41.997Z
paper_authors._id: 6373037cd3ba6bd3f9bc32fa
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6373037cd3ba6bd3f9bc32fa/dIL4AIcL24QL_Rp4B9ewS.png
paper_authors.fullname: Shuai Yang
paper_authors.type: user
paper_authors.user: ysmikey
paper_authors._id: 66cbfc93888e99a28d4724c5
paper_authors.name: Jing Tan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-27T07:42:42.514Z
paper_authors._id: 65367c40061949598892dbdc
paper_authors.avatarUrl: /avatars/4baf27263841471cbd5f629a8b99424d.svg
paper_authors.fullname: Jing Tan
paper_authors.type: user
paper_authors.user: jingtan
paper_authors._id: 66cbfc93888e99a28d4724c6
paper_authors.name: Mengchen Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-11T09:22:08.873Z
paper_authors._id: 64de20c5808492ba6e65d124
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64de20c5808492ba6e65d124/58IX_TI5vJw73qS1knw56.jpeg
paper_authors.fullname: Zhang Mengchen
paper_authors.type: user
paper_authors.user: Dubhe-zmc
paper_authors._id: 66cbfc93888e99a28d4724c7
paper_authors.name: Tong Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-11T09:22:11.294Z
paper_authors._id: 6310929fda0aaad2f70397a1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6310929fda0aaad2f70397a1/YoHeYZezmCOXwL-qsiZOc.png
paper_authors.fullname: Tong Wu
paper_authors.type: user
paper_authors.user: wutong16
paper_authors._id: 66cbfc93888e99a28d4724c8
paper_authors.name: Yixuan Li
paper_authors._id: 66cbfc93888e99a28d4724c9
paper_authors.name: Gordon Wetzstein
paper_authors._id: 66cbfc93888e99a28d4724ca
paper_authors.name: Ziwei Liu
paper_authors._id: 66cbfc93888e99a28d4724cb
paper_authors.name: Dahua Lin
paper_title: LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generation
paper_summary: 3D immersive scene generation is a challenging yet critical task in computer
  vision and graphics. A desired virtual 3D scene should 1) exhibit
  omnidirectional view consistency, and 2) allow for free exploration in complex
  scene hierarchies. Existing methods either rely on successive scene expansion
  via inpainting or employ panorama representation to represent large FOV scene
  environments. However, the generated scene suffers from semantic drift during
  expansion and is unable to handle occlusion among scene hierarchies. To tackle
  these challenges, we introduce LayerPano3D, a novel framework for full-view,
  explorable panoramic 3D scene generation from a single text prompt. Our key
  insight is to decompose a reference 2D panorama into multiple layers at
  different depth levels, where each layer reveals the unseen space from the
  reference views via diffusion prior. LayerPano3D comprises multiple dedicated
  designs: 1) we introduce a novel text-guided anchor view synthesis pipeline for
  high-quality, consistent panorama generation. 2) We pioneer the Layered 3D
  Panorama as underlying representation to manage complex scene hierarchies and
  lift it into 3D Gaussians to splat detailed 360-degree omnidirectional scenes
  with unconstrained viewing paths. Extensive experiments demonstrate that our
  framework generates state-of-the-art 3D panoramic scene in both full view
  consistency and immersive exploratory experience. We believe that LayerPano3D
  holds promise for advancing 3D panoramic scene creation with numerous
  applications.
paper_ai_summary: LayerPano3D generates high-quality, full-view consistent, and immersive 3D panoramic scenes from text using layered diffusion priors and Gaussians.
paper_ai_keywords: omnidirectional view consistency panorama representation scene inpainting semantic drift diffusion prior text-guided anchor view synthesis Layered 3D Panorama 3D Gaussians full-view consistency immersive exploratory experience