title: AutoDecoding Latent 3D Diffusion Models
summary: We present a novel approach to the generation of static and articulated 3D
  assets that has a 3D autodecoder at its core. The 3D autodecoder framework
  embeds properties learned from the target dataset in the latent space, which
  can then be decoded into a volumetric representation for rendering
  view-consistent appearance and geometry. We then identify the appropriate
  intermediate volumetric latent space, and introduce robust normalization and
  de-normalization operations to learn a 3D diffusion from 2D images or monocular
  videos of rigid or articulated objects. Our approach is flexible enough to use
  either existing camera supervision or no camera information at all -- instead
  efficiently learning it during training. Our evaluations demonstrate that our
  generation results outperform state-of-the-art alternatives on various
  benchmark datasets and metrics, including multi-view image datasets of
  synthetic objects, real in-the-wild videos of moving people, and a large-scale,
  real video dataset of static objects.
paper_authors._id: 64ae08835d48838462015149
paper_authors.name: Evangelos Ntavelis
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-13T08:32:42.642Z
paper_authors._id: 64aeca55c693a1954796e404
paper_authors.avatarUrl: /avatars/4d18e907447c2370501e8e3e4088a776.svg
paper_authors.fullname: Evan Ntavelis
paper_authors.type: user
paper_authors.user: entavelis
paper_authors._id: 64ae08835d4883846201514a
paper_authors.name: Aliaksandr Siarohin
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2023-07-12T07:31:58.137Z
paper_authors._id: 64276311eb9a0ed86180715b
paper_authors.avatarUrl: /avatars/76f933cd549f10e5e2db379de235d304.svg
paper_authors.fullname: Aliaksandr Siarohin
paper_authors.type: user
paper_authors.user: aliaksandr-siarohin
paper_authors._id: 64ae08835d4883846201514b
paper_authors.name: Kyle Olszewski
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2023-07-12T04:48:28.327Z
paper_authors._id: 63ccd965336d9d5617164be0
paper_authors.avatarUrl: /avatars/21be405bdbe85d610376520180f7bb41.svg
paper_authors.fullname: Kyle Olszewski
paper_authors.type: user
paper_authors.user: kolsz
paper_authors._id: 64ae08835d4883846201514c
paper_authors.name: Chaoyang Wang
paper_authors._id: 64ae08835d4883846201514d
paper_authors.name: Luc Van Gool
paper_authors._id: 64ae08835d4883846201514e
paper_authors.name: Sergey Tulyakov
paper_title: AutoDecoding Latent 3D Diffusion Models
paper_summary: We present a novel approach to the generation of static and articulated 3D
  assets that has a 3D autodecoder at its core. The 3D autodecoder framework
  embeds properties learned from the target dataset in the latent space, which
  can then be decoded into a volumetric representation for rendering
  view-consistent appearance and geometry. We then identify the appropriate
  intermediate volumetric latent space, and introduce robust normalization and
  de-normalization operations to learn a 3D diffusion from 2D images or monocular
  videos of rigid or articulated objects. Our approach is flexible enough to use
  either existing camera supervision or no camera information at all -- instead
  efficiently learning it during training. Our evaluations demonstrate that our
  generation results outperform state-of-the-art alternatives on various
  benchmark datasets and metrics, including multi-view image datasets of
  synthetic objects, real in-the-wild videos of moving people, and a large-scale,
  real video dataset of static objects.
paper_ai_summary: A 3D autodecoder framework generates high-quality static and articulated 3D assets from 2D images or monocular videos, using latent space embeddings, normalization, and 3D diffusion.
paper_ai_keywords: 3D autodecoder latent space volumetric representation robust normalization de-normalization 3D diffusion multi-view image datasets real in-the-wild videos large-scale real video dataset