MagicFace: High-Fidelity Facial Expression Editing with Action-Unit
  Control
We address the problem of facial expression editing by controling the
relative variation of facial action-unit (AU) from the same person. This
enables us to edit this specific person's expression in a fine-grained,
continuous and interpretable manner, while preserving their identity, pose,
background and detailed facial attributes. Key to our model, which we dub
MagicFace, is a diffusion model conditioned on AU variations and an ID encoder
to preserve facial details of high consistency. Specifically, to preserve the
facial details with the input identity, we leverage the power of pretrained
Stable-Diffusion models and design an ID encoder to merge appearance features
through self-attention. To keep background and pose consistency, we introduce
an efficient Attribute Controller by explicitly informing the model of current
background and pose of the target. By injecting AU variations into a denoising
UNet, our model can animate arbitrary identities with various AU combinations,
yielding superior results in high-fidelity expression editing compared to other
facial expression editing works. Code is publicly available at
https://github.com/weimengting/MagicFace.
677df32e9b2af6b36b22668e
Mengting Wei
admin_assigned
2025-01-08T13:54:19.846Z
6629604bdf81c3e1d90ca5ec
/avatars/bb35569fe124de1374f149b940a4b56c.svg
Mengting Wei
user
mengtingwei
677df32e9b2af6b36b22668f
Tuomas Varanka
677df32e9b2af6b36b226690
Xingxun Jiang
admin_assigned
2025-01-08T13:54:36.033Z
667690daf2312ba18e81a160
/avatars/f502d8ba6cf1de352032e62acab4022d.svg
Xingxun JIANG
user
Xingxun
677df32e9b2af6b36b226691
Huai-Qian Khor
677df32e9b2af6b36b226692
Guoying Zhao
MagicFace: High-Fidelity Facial Expression Editing with Action-Unit
  Control
We address the problem of facial expression editing by controling the
relative variation of facial action-unit (AU) from the same person. This
enables us to edit this specific person's expression in a fine-grained,
continuous and interpretable manner, while preserving their identity, pose,
background and detailed facial attributes. Key to our model, which we dub
MagicFace, is a diffusion model conditioned on AU variations and an ID encoder
to preserve facial details of high consistency. Specifically, to preserve the
facial details with the input identity, we leverage the power of pretrained
Stable-Diffusion models and design an ID encoder to merge appearance features
through self-attention. To keep background and pose consistency, we introduce
an efficient Attribute Controller by explicitly informing the model of current
background and pose of the target. By injecting AU variations into a denoising
UNet, our model can animate arbitrary identities with various AU combinations,
yielding superior results in high-fidelity expression editing compared to other
facial expression editing works. Code is publicly available at
https://github.com/weimengting/MagicFace.
MagicFace uses a diffusion model and an ID encoder with self-attention to edit facial expressions while preserving identity, pose, background, and attributes, achieving high-fidelity results.
diffusion model ID encoder self-attention facial expression editing facial action-unit denoising UNet Attribute Controller