title: SpaceBlender: Creating Context-Rich Collaborative Spaces Through
    Generative 3D Scene Blending
summary: There is increased interest in using generative AI to create 3D spaces for
  Virtual Reality (VR) applications. However, today's models produce artificial
  environments, falling short of supporting collaborative tasks that benefit from
  incorporating the user's physical context. To generate environments that
  support VR telepresence, we introduce SpaceBlender, a novel pipeline that
  utilizes generative AI techniques to blend users' physical surroundings into
  unified virtual spaces. This pipeline transforms user-provided 2D images into
  context-rich 3D environments through an iterative process consisting of depth
  estimation, mesh alignment, and diffusion-based space completion guided by
  geometric priors and adaptive text prompts. In a preliminary within-subjects
  study, where 20 participants performed a collaborative VR affinity diagramming
  task in pairs, we compared SpaceBlender with a generic virtual environment and
  a state-of-the-art scene generation framework, evaluating its ability to create
  virtual spaces suitable for collaboration. Participants appreciated the
  enhanced familiarity and context provided by SpaceBlender but also noted
  complexities in the generative environments that could detract from task focus.
  Drawing on participant feedback, we propose directions for improving the
  pipeline and discuss the value and design of blended spaces for different
  scenarios.
paper_authors._id: 66f29ac6ba35495ded05a1dd
paper_authors.name: Nels Numan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-24T14:08:25.428Z
paper_authors._id: 64bb15e45c457ccaa4dde0a3
paper_authors.avatarUrl: /avatars/86485ae198880c4b774eda7e85642beb.svg
paper_authors.fullname: Nels
paper_authors.type: user
paper_authors.user: nels-numan
paper_authors._id: 66f29ac6ba35495ded05a1de
paper_authors.name: Shwetha Rajaram
paper_authors._id: 66f29ac6ba35495ded05a1df
paper_authors.name: Balasaravanan Thoravi Kumaravel
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-09-24T10:56:08.838Z
paper_authors._id: 6389163dec1f539adc0c33dc
paper_authors.avatarUrl: /avatars/c737632ad56030991d1dc781438566c4.svg
paper_authors.fullname: Thor Thunder
paper_authors.type: user
paper_authors.user: thorthund3r
paper_authors._id: 66f29ac6ba35495ded05a1e0
paper_authors.name: Nicolai Marquardt
paper_authors._id: 66f29ac6ba35495ded05a1e1
paper_authors.name: Andrew D. Wilson
paper_title: SpaceBlender: Creating Context-Rich Collaborative Spaces Through
    Generative 3D Scene Blending
paper_summary: There is increased interest in using generative AI to create 3D spaces for
  Virtual Reality (VR) applications. However, today's models produce artificial
  environments, falling short of supporting collaborative tasks that benefit from
  incorporating the user's physical context. To generate environments that
  support VR telepresence, we introduce SpaceBlender, a novel pipeline that
  utilizes generative AI techniques to blend users' physical surroundings into
  unified virtual spaces. This pipeline transforms user-provided 2D images into
  context-rich 3D environments through an iterative process consisting of depth
  estimation, mesh alignment, and diffusion-based space completion guided by
  geometric priors and adaptive text prompts. In a preliminary within-subjects
  study, where 20 participants performed a collaborative VR affinity diagramming
  task in pairs, we compared SpaceBlender with a generic virtual environment and
  a state-of-the-art scene generation framework, evaluating its ability to create
  virtual spaces suitable for collaboration. Participants appreciated the
  enhanced familiarity and context provided by SpaceBlender but also noted
  complexities in the generative environments that could detract from task focus.
  Drawing on participant feedback, we propose directions for improving the
  pipeline and discuss the value and design of blended spaces for different
  scenarios.
paper_ai_summary: SpaceBlender, a generative AI pipeline, integrates user's physical surroundings into VR environments through depth estimation, mesh alignment, and diffusion-based completion, enhancing collaborative VR experiences.
paper_ai_keywords: generative AI VR telepresence SpaceBlender depth estimation mesh alignment diffusion-based space completion geometric priors adaptive text prompts VR affinity diagramming