title: MagicFace: High-Fidelity Facial Expression Editing with Action-Unit
    Control
summary: We address the problem of facial expression editing by controling the
  relative variation of facial action-unit (AU) from the same person. This
  enables us to edit this specific person's expression in a fine-grained,
  continuous and interpretable manner, while preserving their identity, pose,
  background and detailed facial attributes. Key to our model, which we dub
  MagicFace, is a diffusion model conditioned on AU variations and an ID encoder
  to preserve facial details of high consistency. Specifically, to preserve the
  facial details with the input identity, we leverage the power of pretrained
  Stable-Diffusion models and design an ID encoder to merge appearance features
  through self-attention. To keep background and pose consistency, we introduce
  an efficient Attribute Controller by explicitly informing the model of current
  background and pose of the target. By injecting AU variations into a denoising
  UNet, our model can animate arbitrary identities with various AU combinations,
  yielding superior results in high-fidelity expression editing compared to other
  facial expression editing works. Code is publicly available at
  https://github.com/weimengting/MagicFace.
paper_authors._id: 677df32e9b2af6b36b22668e
paper_authors.name: Mengting Wei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-08T13:54:19.846Z
paper_authors._id: 6629604bdf81c3e1d90ca5ec
paper_authors.avatarUrl: /avatars/bb35569fe124de1374f149b940a4b56c.svg
paper_authors.fullname: Mengting Wei
paper_authors.type: user
paper_authors.user: mengtingwei
paper_authors._id: 677df32e9b2af6b36b22668f
paper_authors.name: Tuomas Varanka
paper_authors._id: 677df32e9b2af6b36b226690
paper_authors.name: Xingxun Jiang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-08T13:54:36.033Z
paper_authors._id: 667690daf2312ba18e81a160
paper_authors.avatarUrl: /avatars/f502d8ba6cf1de352032e62acab4022d.svg
paper_authors.fullname: Xingxun JIANG
paper_authors.type: user
paper_authors.user: Xingxun
paper_authors._id: 677df32e9b2af6b36b226691
paper_authors.name: Huai-Qian Khor
paper_authors._id: 677df32e9b2af6b36b226692
paper_authors.name: Guoying Zhao
paper_title: MagicFace: High-Fidelity Facial Expression Editing with Action-Unit
    Control
paper_summary: We address the problem of facial expression editing by controling the
  relative variation of facial action-unit (AU) from the same person. This
  enables us to edit this specific person's expression in a fine-grained,
  continuous and interpretable manner, while preserving their identity, pose,
  background and detailed facial attributes. Key to our model, which we dub
  MagicFace, is a diffusion model conditioned on AU variations and an ID encoder
  to preserve facial details of high consistency. Specifically, to preserve the
  facial details with the input identity, we leverage the power of pretrained
  Stable-Diffusion models and design an ID encoder to merge appearance features
  through self-attention. To keep background and pose consistency, we introduce
  an efficient Attribute Controller by explicitly informing the model of current
  background and pose of the target. By injecting AU variations into a denoising
  UNet, our model can animate arbitrary identities with various AU combinations,
  yielding superior results in high-fidelity expression editing compared to other
  facial expression editing works. Code is publicly available at
  https://github.com/weimengting/MagicFace.
paper_ai_keywords: diffusion model AU variations ID encoder Stable-Diffusion models self-attention Attribute Controller denoising UNet high-fidelity expression editing