The KL-autoencoder used in our model is a variational
autoencoder (VAE) architecture based on [7], which in-
corporates a KL divergence loss term.
this
model for our speciﬁc needs, we modiﬁed the ﬁrst and last
Conv2d layers of the KL-autoencoder. These adjustments
allowed the model to accommodate the modiﬁed input for-
mat, which consists of concatenated RGB images and depth
maps.

To adapt

The generative diffusion model utilizes a U-Net back-
bone [21] architecture, primarily composed of 2D con-
volutional layers. The diffusion model was trained on
the learned, low-dimensional, KL-regularized latent space,
similar to [20].
Enabling more accurate reconstruc-
tions and efﬁcient high-resolution synthesis, compared to
transformer-based diffusion model trained in pixel space.
LDM3D- Latent Diffusion Model for 3D