title: Single-Shot Implicit Morphable Faces with Consistent Texture
    Parameterization
summary: There is a growing demand for the accessible creation of high-quality 3D
  avatars that are animatable and customizable. Although 3D morphable models
  provide intuitive control for editing and animation, and robustness for
  single-view face reconstruction, they cannot easily capture geometric and
  appearance details. Methods based on neural implicit representations, such as
  signed distance functions (SDF) or neural radiance fields, approach
  photo-realism, but are difficult to animate and do not generalize well to
  unseen data. To tackle this problem, we propose a novel method for constructing
  implicit 3D morphable face models that are both generalizable and intuitive for
  editing. Trained from a collection of high-quality 3D scans, our face model is
  parameterized by geometry, expression, and texture latent codes with a learned
  SDF and explicit UV texture parameterization. Once trained, we can reconstruct
  an avatar from a single in-the-wild image by leveraging the learned prior to
  project the image into the latent space of our model. Our implicit morphable
  face models can be used to render an avatar from novel views, animate facial
  expressions by modifying expression codes, and edit textures by directly
  painting on the learned UV-texture maps. We demonstrate quantitatively and
  qualitatively that our method improves upon photo-realism, geometry, and
  expression accuracy compared to state-of-the-art methods.
paper_authors._id: 6454883ccd09ceba0e170244
paper_authors.name: Connor Z. Lin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-26T08:07:33.909Z
paper_authors._id: 646fded8dcddb2358a277bac
paper_authors.avatarUrl: /avatars/1a0afd397cbc6902d1b9dbe20bc71cb7.svg
paper_authors.fullname: Connor Zhizhen Lin
paper_authors.type: user
paper_authors.user: connorzl
paper_authors._id: 6454883ccd09ceba0e170245
paper_authors.name: Koki Nagano
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-05T12:56:59.876Z
paper_authors._id: 637eb1c71dbae09191059243
paper_authors.avatarUrl: /avatars/674b1a6e419c758a8613338d20f21544.svg
paper_authors.fullname: Koki Nagano
paper_authors.type: user
paper_authors.user: luminohope
paper_authors._id: 6454883ccd09ceba0e170246
paper_authors.name: Jan Kautz
paper_authors._id: 6454883ccd09ceba0e170247
paper_authors.name: Eric R. Chan
paper_authors._id: 6454883ccd09ceba0e170248
paper_authors.name: Umar Iqbal
paper_authors._id: 6454883ccd09ceba0e170249
paper_authors.name: Leonidas Guibas
paper_authors._id: 6454883ccd09ceba0e17024a
paper_authors.name: Gordon Wetzstein
paper_authors._id: 6454883ccd09ceba0e17024b
paper_authors.name: Sameh Khamis
paper_title: Single-Shot Implicit Morphable Faces with Consistent Texture
    Parameterization
paper_summary: There is a growing demand for the accessible creation of high-quality 3D
  avatars that are animatable and customizable. Although 3D morphable models
  provide intuitive control for editing and animation, and robustness for
  single-view face reconstruction, they cannot easily capture geometric and
  appearance details. Methods based on neural implicit representations, such as
  signed distance functions (SDF) or neural radiance fields, approach
  photo-realism, but are difficult to animate and do not generalize well to
  unseen data. To tackle this problem, we propose a novel method for constructing
  implicit 3D morphable face models that are both generalizable and intuitive for
  editing. Trained from a collection of high-quality 3D scans, our face model is
  parameterized by geometry, expression, and texture latent codes with a learned
  SDF and explicit UV texture parameterization. Once trained, we can reconstruct
  an avatar from a single in-the-wild image by leveraging the learned prior to
  project the image into the latent space of our model. Our implicit morphable
  face models can be used to render an avatar from novel views, animate facial
  expressions by modifying expression codes, and edit textures by directly
  painting on the learned UV-texture maps. We demonstrate quantitatively and
  qualitatively that our method improves upon photo-realism, geometry, and
  expression accuracy compared to state-of-the-art methods.
paper_ai_summary: A novel method for constructing implicit 3D morphable face models that combines the benefits of neural implicit representations and 3D morphable models, offering photo-realism, generalizability, and intuitive editing capabilities.
paper_ai_keywords: 3D morphable models neural implicit representations signed distance functions neural radiance fields latent codes UV texture parameterization single-view face reconstruction photo-realism facial expressions texture editing