FitMe: Deep Photorealistic 3D Morphable Model Avatars
In this paper, we introduce FitMe, a facial reflectance model and a
differentiable rendering optimization pipeline, that can be used to acquire
high-fidelity renderable human avatars from single or multiple images. The
model consists of a multi-modal style-based generator, that captures facial
appearance in terms of diffuse and specular reflectance, and a PCA-based shape
model. We employ a fast differentiable rendering process that can be used in an
optimization pipeline, while also achieving photorealistic facial shading. Our
optimization process accurately captures both the facial reflectance and shape
in high-detail, by exploiting the expressivity of the style-based latent
representation and of our shape model. FitMe achieves state-of-the-art
reflectance acquisition and identity preservation on single "in-the-wild"
facial images, while it produces impressive scan-like results, when given
multiple unconstrained facial images pertaining to the same identity. In
contrast with recent implicit avatar reconstructions, FitMe requires only one
minute and produces relightable mesh and texture-based avatars, that can be
used by end-user applications.
64644326823c7f8805514e66
Alexandros Lattas
claimed_verified
2023-05-17T08:03:48.698Z
645df1be8a22774edb290765
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/OduGNNzXnhkwnPkBZUw3B.jpeg
Alexandros Lattas
user
lattas
64644326823c7f8805514e67
Stylianos Moschoglou
64644326823c7f8805514e68
Stylianos Ploumpis
64644326823c7f8805514e69
Baris Gecer
64644326823c7f8805514e6a
Jiankang Deng
claimed_verified
2023-05-19T10:28:15.581Z
62cc7a38376917c0223dd24b
https://cdn-avatars.huggingface.co/v1/production/uploads/1657566065867-noauth.png
JiankangDeng
user
JiankangDeng
64644326823c7f8805514e6b
Stefanos Zafeiriou
FitMe: Deep Photorealistic 3D Morphable Model Avatars
In this paper, we introduce FitMe, a facial reflectance model and a
differentiable rendering optimization pipeline, that can be used to acquire
high-fidelity renderable human avatars from single or multiple images. The
model consists of a multi-modal style-based generator, that captures facial
appearance in terms of diffuse and specular reflectance, and a PCA-based shape
model. We employ a fast differentiable rendering process that can be used in an
optimization pipeline, while also achieving photorealistic facial shading. Our
optimization process accurately captures both the facial reflectance and shape
in high-detail, by exploiting the expressivity of the style-based latent
representation and of our shape model. FitMe achieves state-of-the-art
reflectance acquisition and identity preservation on single "in-the-wild"
facial images, while it produces impressive scan-like results, when given
multiple unconstrained facial images pertaining to the same identity. In
contrast with recent implicit avatar reconstructions, FitMe requires only one
minute and produces relightable mesh and texture-based avatars, that can be
used by end-user applications.
FitMe is a facial reflectance model with a differentiable rendering pipeline that generates high-fidelity human avatars from single or multiple images, offering accurate reflectance and identity preservation.
facial reflectance model differentiable rendering optimization pipeline multi-modal style-based generator diffuse and specular reflectance PCA-based shape model differentiable rendering style-based latent representation photorealistic facial shading relightable mesh and texture-based avatars