title: FitMe: Deep Photorealistic 3D Morphable Model Avatars
summary: In this paper, we introduce FitMe, a facial reflectance model and a
  differentiable rendering optimization pipeline, that can be used to acquire
  high-fidelity renderable human avatars from single or multiple images. The
  model consists of a multi-modal style-based generator, that captures facial
  appearance in terms of diffuse and specular reflectance, and a PCA-based shape
  model. We employ a fast differentiable rendering process that can be used in an
  optimization pipeline, while also achieving photorealistic facial shading. Our
  optimization process accurately captures both the facial reflectance and shape
  in high-detail, by exploiting the expressivity of the style-based latent
  representation and of our shape model. FitMe achieves state-of-the-art
  reflectance acquisition and identity preservation on single "in-the-wild"
  facial images, while it produces impressive scan-like results, when given
  multiple unconstrained facial images pertaining to the same identity. In
  contrast with recent implicit avatar reconstructions, FitMe requires only one
  minute and produces relightable mesh and texture-based avatars, that can be
  used by end-user applications.
paper_authors._id: 64644326823c7f8805514e66
paper_authors.name: Alexandros Lattas
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-17T08:03:48.698Z
paper_authors._id: 645df1be8a22774edb290765
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/OduGNNzXnhkwnPkBZUw3B.jpeg
paper_authors.fullname: Alexandros Lattas
paper_authors.type: user
paper_authors.user: lattas
paper_authors._id: 64644326823c7f8805514e67
paper_authors.name: Stylianos Moschoglou
paper_authors._id: 64644326823c7f8805514e68
paper_authors.name: Stylianos Ploumpis
paper_authors._id: 64644326823c7f8805514e69
paper_authors.name: Baris Gecer
paper_authors._id: 64644326823c7f8805514e6a
paper_authors.name: Jiankang Deng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-19T10:28:15.581Z
paper_authors._id: 62cc7a38376917c0223dd24b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1657566065867-noauth.png
paper_authors.fullname: JiankangDeng
paper_authors.type: user
paper_authors.user: JiankangDeng
paper_authors._id: 64644326823c7f8805514e6b
paper_authors.name: Stefanos Zafeiriou
paper_title: FitMe: Deep Photorealistic 3D Morphable Model Avatars
paper_summary: In this paper, we introduce FitMe, a facial reflectance model and a
  differentiable rendering optimization pipeline, that can be used to acquire
  high-fidelity renderable human avatars from single or multiple images. The
  model consists of a multi-modal style-based generator, that captures facial
  appearance in terms of diffuse and specular reflectance, and a PCA-based shape
  model. We employ a fast differentiable rendering process that can be used in an
  optimization pipeline, while also achieving photorealistic facial shading. Our
  optimization process accurately captures both the facial reflectance and shape
  in high-detail, by exploiting the expressivity of the style-based latent
  representation and of our shape model. FitMe achieves state-of-the-art
  reflectance acquisition and identity preservation on single "in-the-wild"
  facial images, while it produces impressive scan-like results, when given
  multiple unconstrained facial images pertaining to the same identity. In
  contrast with recent implicit avatar reconstructions, FitMe requires only one
  minute and produces relightable mesh and texture-based avatars, that can be
  used by end-user applications.
paper_ai_keywords: facial reflectance model differentiable rendering optimization pipeline multi-modal style-based generator diffuse reflectance specular reflectance PCA-based shape model differentiable rendering process optimization pipeline photorealistic facial shading style-based latent representation relightable mesh texture-based avatars