SpaceBlender: Creating Context-Rich Collaborative Spaces Through
  Generative 3D Scene Blending
There is increased interest in using generative AI to create 3D spaces for
Virtual Reality (VR) applications. However, today's models produce artificial
environments, falling short of supporting collaborative tasks that benefit from
incorporating the user's physical context. To generate environments that
support VR telepresence, we introduce SpaceBlender, a novel pipeline that
utilizes generative AI techniques to blend users' physical surroundings into
unified virtual spaces. This pipeline transforms user-provided 2D images into
context-rich 3D environments through an iterative process consisting of depth
estimation, mesh alignment, and diffusion-based space completion guided by
geometric priors and adaptive text prompts. In a preliminary within-subjects
study, where 20 participants performed a collaborative VR affinity diagramming
task in pairs, we compared SpaceBlender with a generic virtual environment and
a state-of-the-art scene generation framework, evaluating its ability to create
virtual spaces suitable for collaboration. Participants appreciated the
enhanced familiarity and context provided by SpaceBlender but also noted
complexities in the generative environments that could detract from task focus.
Drawing on participant feedback, we propose directions for improving the
pipeline and discuss the value and design of blended spaces for different
scenarios.
66f29ac6ba35495ded05a1dd
Nels Numan
claimed_verified
2024-09-24T14:08:25.428Z
64bb15e45c457ccaa4dde0a3
/avatars/86485ae198880c4b774eda7e85642beb.svg
Nels
user
nels-numan
66f29ac6ba35495ded05a1de
Shwetha Rajaram
66f29ac6ba35495ded05a1df
Balasaravanan Thoravi Kumaravel
extracted_pending
2024-09-24T10:56:08.838Z
6389163dec1f539adc0c33dc
/avatars/c737632ad56030991d1dc781438566c4.svg
Thor Thunder
user
thorthund3r
66f29ac6ba35495ded05a1e0
Nicolai Marquardt
66f29ac6ba35495ded05a1e1
Andrew D. Wilson
SpaceBlender: Creating Context-Rich Collaborative Spaces Through
  Generative 3D Scene Blending
There is increased interest in using generative AI to create 3D spaces for
Virtual Reality (VR) applications. However, today's models produce artificial
environments, falling short of supporting collaborative tasks that benefit from
incorporating the user's physical context. To generate environments that
support VR telepresence, we introduce SpaceBlender, a novel pipeline that
utilizes generative AI techniques to blend users' physical surroundings into
unified virtual spaces. This pipeline transforms user-provided 2D images into
context-rich 3D environments through an iterative process consisting of depth
estimation, mesh alignment, and diffusion-based space completion guided by
geometric priors and adaptive text prompts. In a preliminary within-subjects
study, where 20 participants performed a collaborative VR affinity diagramming
task in pairs, we compared SpaceBlender with a generic virtual environment and
a state-of-the-art scene generation framework, evaluating its ability to create
virtual spaces suitable for collaboration. Participants appreciated the
enhanced familiarity and context provided by SpaceBlender but also noted
complexities in the generative environments that could detract from task focus.
Drawing on participant feedback, we propose directions for improving the
pipeline and discuss the value and design of blended spaces for different
scenarios.
SpaceBlender, a generative AI pipeline, integrates user's physical surroundings into VR environments through depth estimation, mesh alignment, and diffusion-based completion, enhancing collaborative VR experiences.
generative AI VR telepresence SpaceBlender depth estimation mesh alignment diffusion-based space completion geometric priors adaptive text prompts VR affinity diagramming