title: DreamSpace: Dreaming Your Room Space with Text-Driven Panoramic Texture
    Propagation
summary: Diffusion-based methods have achieved prominent success in generating 2D
  media. However, accomplishing similar proficiencies for scene-level mesh
  texturing in 3D spatial applications, e.g., XR/VR, remains constrained,
  primarily due to the intricate nature of 3D geometry and the necessity for
  immersive free-viewpoint rendering. In this paper, we propose a novel indoor
  scene texturing framework, which delivers text-driven texture generation with
  enchanting details and authentic spatial coherence. The key insight is to first
  imagine a stylized 360{\deg} panoramic texture from the central viewpoint of
  the scene, and then propagate it to the rest areas with inpainting and
  imitating techniques. To ensure meaningful and aligned textures to the scene,
  we develop a novel coarse-to-fine panoramic texture generation approach with
  dual texture alignment, which both considers the geometry and texture cues of
  the captured scenes. To survive from cluttered geometries during texture
  propagation, we design a separated strategy, which conducts texture inpainting
  in confidential regions and then learns an implicit imitating network to
  synthesize textures in occluded and tiny structural areas. Extensive
  experiments and the immersive VR application on real-world indoor scenes
  demonstrate the high quality of the generated textures and the engaging
  experience on VR headsets. Project webpage:
  https://ybbbbt.com/publication/dreamspace
paper_authors._id: 6535f496ae42162a1e110ea1
paper_authors.name: Bangbang Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-23T09:06:23.653Z
paper_authors._id: 63d748ff6f49aa82306b7e48
paper_authors.avatarUrl: /avatars/9f0b8b8a09b14d76e52ed1bd312e6b63.svg
paper_authors.fullname: BB Yang
paper_authors.type: user
paper_authors.user: ybbbbt
paper_authors._id: 6535f496ae42162a1e110ea2
paper_authors.name: Wenqi Dong
paper_authors._id: 6535f496ae42162a1e110ea3
paper_authors.name: Lin Ma
paper_authors._id: 6535f496ae42162a1e110ea4
paper_authors.name: Wenbo Hu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-04T12:00:55.558Z
paper_authors._id: 657a7458afbb0117ba15c59f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/657a7458afbb0117ba15c59f/8_iwTS1UG_mKnfylFbLsY.jpeg
paper_authors.fullname: Wenbo Hu
paper_authors.type: user
paper_authors.user: wbhu-tc
paper_authors._id: 6535f496ae42162a1e110ea5
paper_authors.name: Xiao Liu
paper_authors._id: 6535f496ae42162a1e110ea6
paper_authors.name: Zhaopeng Cui
paper_authors._id: 6535f496ae42162a1e110ea7
paper_authors.name: Yuewen Ma
paper_title: DreamSpace: Dreaming Your Room Space with Text-Driven Panoramic Texture
    Propagation
paper_summary: Diffusion-based methods have achieved prominent success in generating 2D
  media. However, accomplishing similar proficiencies for scene-level mesh
  texturing in 3D spatial applications, e.g., XR/VR, remains constrained,
  primarily due to the intricate nature of 3D geometry and the necessity for
  immersive free-viewpoint rendering. In this paper, we propose a novel indoor
  scene texturing framework, which delivers text-driven texture generation with
  enchanting details and authentic spatial coherence. The key insight is to first
  imagine a stylized 360{\deg} panoramic texture from the central viewpoint of
  the scene, and then propagate it to the rest areas with inpainting and
  imitating techniques. To ensure meaningful and aligned textures to the scene,
  we develop a novel coarse-to-fine panoramic texture generation approach with
  dual texture alignment, which both considers the geometry and texture cues of
  the captured scenes. To survive from cluttered geometries during texture
  propagation, we design a separated strategy, which conducts texture inpainting
  in confidential regions and then learns an implicit imitating network to
  synthesize textures in occluded and tiny structural areas. Extensive
  experiments and the immersive VR application on real-world indoor scenes
  demonstrate the high quality of the generated textures and the engaging
  experience on VR headsets. Project webpage:
  https://ybbbbt.com/publication/dreamspace
paper_ai_summary: A novel indoor scene texturing framework uses diffusion-based methods for text-driven texture generation with high detail and spatial coherence, addressing challenges in 3D spatial applications like XR/VR through dual texture alignment and a separated inpainting strategy.
paper_ai_keywords: diffusion-based methods scene-level mesh texturing 3D spatial applications scene texturing framework stylized 360{\deg} panoramic texture inpainting techniques dual texture alignment texture inpainting implicit imitating network real-world indoor scenes VR application