Single-Shot Implicit Morphable Faces with Consistent Texture
  Parameterization
There is a growing demand for the accessible creation of high-quality 3D
avatars that are animatable and customizable. Although 3D morphable models
provide intuitive control for editing and animation, and robustness for
single-view face reconstruction, they cannot easily capture geometric and
appearance details. Methods based on neural implicit representations, such as
signed distance functions (SDF) or neural radiance fields, approach
photo-realism, but are difficult to animate and do not generalize well to
unseen data. To tackle this problem, we propose a novel method for constructing
implicit 3D morphable face models that are both generalizable and intuitive for
editing. Trained from a collection of high-quality 3D scans, our face model is
parameterized by geometry, expression, and texture latent codes with a learned
SDF and explicit UV texture parameterization. Once trained, we can reconstruct
an avatar from a single in-the-wild image by leveraging the learned prior to
project the image into the latent space of our model. Our implicit morphable
face models can be used to render an avatar from novel views, animate facial
expressions by modifying expression codes, and edit textures by directly
painting on the learned UV-texture maps. We demonstrate quantitatively and
qualitatively that our method improves upon photo-realism, geometry, and
expression accuracy compared to state-of-the-art methods.
6454883ccd09ceba0e170244
Connor Z. Lin
claimed_verified
2023-05-26T08:07:33.909Z
646fded8dcddb2358a277bac
/avatars/1a0afd397cbc6902d1b9dbe20bc71cb7.svg
Connor Zhizhen Lin
user
connorzl
6454883ccd09ceba0e170245
Koki Nagano
claimed_verified
2023-05-05T12:56:59.876Z
637eb1c71dbae09191059243
/avatars/674b1a6e419c758a8613338d20f21544.svg
Koki Nagano
user
luminohope
6454883ccd09ceba0e170246
Jan Kautz
6454883ccd09ceba0e170247
Eric R. Chan
6454883ccd09ceba0e170248
Umar Iqbal
6454883ccd09ceba0e170249
Leonidas Guibas
6454883ccd09ceba0e17024a
Gordon Wetzstein
6454883ccd09ceba0e17024b
Sameh Khamis
Single-Shot Implicit Morphable Faces with Consistent Texture
  Parameterization
There is a growing demand for the accessible creation of high-quality 3D
avatars that are animatable and customizable. Although 3D morphable models
provide intuitive control for editing and animation, and robustness for
single-view face reconstruction, they cannot easily capture geometric and
appearance details. Methods based on neural implicit representations, such as
signed distance functions (SDF) or neural radiance fields, approach
photo-realism, but are difficult to animate and do not generalize well to
unseen data. To tackle this problem, we propose a novel method for constructing
implicit 3D morphable face models that are both generalizable and intuitive for
editing. Trained from a collection of high-quality 3D scans, our face model is
parameterized by geometry, expression, and texture latent codes with a learned
SDF and explicit UV texture parameterization. Once trained, we can reconstruct
an avatar from a single in-the-wild image by leveraging the learned prior to
project the image into the latent space of our model. Our implicit morphable
face models can be used to render an avatar from novel views, animate facial
expressions by modifying expression codes, and edit textures by directly
painting on the learned UV-texture maps. We demonstrate quantitatively and
qualitatively that our method improves upon photo-realism, geometry, and
expression accuracy compared to state-of-the-art methods.
A novel method for constructing implicit 3D morphable face models that combines the benefits of neural implicit representations and 3D morphable models, offering photo-realism, generalizability, and intuitive editing capabilities.
3D morphable models neural implicit representations signed distance functions neural radiance fields latent codes UV texture parameterization single-view face reconstruction photo-realism facial expressions texture editing