title: LDM3D-VR: Latent Diffusion Model for 3D VR
summary: Latent diffusion models have proven to be state-of-the-art in the creation
  and manipulation of visual outputs. However, as far as we know, the generation
  of depth maps jointly with RGB is still limited. We introduce LDM3D-VR, a suite
  of diffusion models targeting virtual reality development that includes
  LDM3D-pano and LDM3D-SR. These models enable the generation of panoramic RGBD
  based on textual prompts and the upscaling of low-resolution inputs to
  high-resolution RGBD, respectively. Our models are fine-tuned from existing
  pretrained models on datasets containing panoramic/high-resolution RGB images,
  depth maps and captions. Both models are evaluated in comparison to existing
  related methods.
paper_authors._id: 6549e852fc80ab2d8ca40a20
paper_authors.name: Gabriela Ben Melech Stan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-11-09T09:20:20.208Z
paper_authors._id: 63174cfddc97a974718f373a
paper_authors.avatarUrl: /avatars/f5a1d5d90abf8aafe8864bc4f1b076cc.svg
paper_authors.fullname: gabriela ben melech
paper_authors.type: user
paper_authors.user: gabsm
paper_authors._id: 6549e852fc80ab2d8ca40a21
paper_authors.name: Diana Wofk
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-04T07:26:28.561Z
paper_authors._id: 66d740a519214d743cc30335
paper_authors.avatarUrl: /avatars/bc0839af30a183d5fae0a353d02733e2.svg
paper_authors.fullname: Diana Wofk
paper_authors.type: user
paper_authors.user: dwofk
paper_authors._id: 6549e852fc80ab2d8ca40a22
paper_authors.name: Estelle Aflalo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-07T16:13:42.992Z
paper_authors._id: 64575266900b28f7841090e5
paper_authors.avatarUrl: /avatars/13dc0a370bb24271653c352897a2673e.svg
paper_authors.fullname: Estelle Aflalo
paper_authors.type: user
paper_authors.user: estellea
paper_authors._id: 6549e852fc80ab2d8ca40a23
paper_authors.name: Shao-Yen Tseng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-11-07T17:12:46.489Z
paper_authors._id: 6316524d76c93113fb1572ac
paper_authors.avatarUrl: /avatars/39824232871f0a4b402d4e89435da58f.svg
paper_authors.fullname: Shaoyen Tseng
paper_authors.type: user
paper_authors.user: shaoyent
paper_authors._id: 6549e852fc80ab2d8ca40a24
paper_authors.name: Zhipeng Cai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-11-07T20:23:35.699Z
paper_authors._id: 6419fe794577db917f6d17fc
paper_authors.avatarUrl: /avatars/ed10371a0082c5e02d32b76217200207.svg
paper_authors.fullname: zhipeng cai
paper_authors.type: user
paper_authors.user: zcai
paper_authors._id: 6549e852fc80ab2d8ca40a25
paper_authors.name: Michael Paulitsch
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-07T16:14:06.607Z
paper_authors._id: 645e367e1db05f131837ded6
paper_authors.avatarUrl: /avatars/c8882aa99f14fe872dabd7f2d25b4d38.svg
paper_authors.fullname: Michael Paulitsch
paper_authors.type: user
paper_authors.user: Paulitsch
paper_authors._id: 6549e852fc80ab2d8ca40a26
paper_authors.name: Vasudev Lal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-07T16:14:13.593Z
paper_authors._id: 6454021413dba495b2dfebd1
paper_authors.avatarUrl: /avatars/16f019f7c321aa24000364a50a1975fe.svg
paper_authors.fullname: Vasudev Lal
paper_authors.type: user
paper_authors.user: vasudevlal
paper_title: LDM3D-VR: Latent Diffusion Model for 3D VR
paper_summary: Latent diffusion models have proven to be state-of-the-art in the creation
  and manipulation of visual outputs. However, as far as we know, the generation
  of depth maps jointly with RGB is still limited. We introduce LDM3D-VR, a suite
  of diffusion models targeting virtual reality development that includes
  LDM3D-pano and LDM3D-SR. These models enable the generation of panoramic RGBD
  based on textual prompts and the upscaling of low-resolution inputs to
  high-resolution RGBD, respectively. Our models are fine-tuned from existing
  pretrained models on datasets containing panoramic/high-resolution RGB images,
  depth maps and captions. Both models are evaluated in comparison to existing
  related methods.
paper_ai_summary: LDM3D-VR, comprising LDM3D-pano and LDM3D-SR, generates high-resolution RGBD from textual prompts and low-resolution inputs, demonstrating state-of-the-art performance in virtual reality applications.
paper_ai_keywords: latent diffusion models RGBD virtual reality LDM3D-VR LDM3D-pano LDM3D-SR pretrained models datasets depth maps captions