title: DreamActor-M1: Holistic, Expressive and Robust Human Image Animation
    with Hybrid Guidance
summary: While recent image-based human animation methods achieve realistic body and
  facial motion synthesis, critical gaps remain in fine-grained holistic
  controllability, multi-scale adaptability, and long-term temporal coherence,
  which leads to their lower expressiveness and robustness. We propose a
  diffusion transformer (DiT) based framework, DreamActor-M1, with hybrid
  guidance to overcome these limitations. For motion guidance, our hybrid control
  signals that integrate implicit facial representations, 3D head spheres, and 3D
  body skeletons achieve robust control of facial expressions and body movements,
  while producing expressive and identity-preserving animations. For scale
  adaptation, to handle various body poses and image scales ranging from
  portraits to full-body views, we employ a progressive training strategy using
  data with varying resolutions and scales. For appearance guidance, we integrate
  motion patterns from sequential frames with complementary visual references,
  ensuring long-term temporal coherence for unseen regions during complex
  movements. Experiments demonstrate that our method outperforms the
  state-of-the-art works, delivering expressive results for portraits,
  upper-body, and full-body generation with robust long-term consistency. Project
  Page: https://grisoon.github.io/DreamActor-M1/.
paper_authors._id: 67edf7b6d277de0ec2aa5b6b
paper_authors.name: Yuxuan Luo
paper_authors._id: 67edf7b6d277de0ec2aa5b6c
paper_authors.name: Zhengkun Rong
paper_authors._id: 67edf7b6d277de0ec2aa5b6d
paper_authors.name: Lizhen Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-03T13:47:42.725Z
paper_authors._id: 63451d2dfeba4bdba59cf9b1
paper_authors.avatarUrl: /avatars/ff5dd2f3b502c54e7c3e2512c3e98b28.svg
paper_authors.fullname: Lizhen Wang
paper_authors.type: user
paper_authors.user: wanglz14
paper_authors._id: 67edf7b6d277de0ec2aa5b6e
paper_authors.name: Longhao Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-03T13:47:48.593Z
paper_authors._id: 602e78801f993496bc14d9c9
paper_authors.avatarUrl: /avatars/eff73f60e6652d107cebeda3b2b87fe1.svg
paper_authors.fullname: Longhao Zhang
paper_authors.type: user
paper_authors.user: zhanglonghao
paper_authors._id: 67edf7b6d277de0ec2aa5b6f
paper_authors.name: Tianshu Hu
paper_authors._id: 67edf7b6d277de0ec2aa5b70
paper_authors.name: Yongming Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-03T13:48:03.944Z
paper_authors._id: 668f66c56575435392165c25
paper_authors.avatarUrl: /avatars/4ebe10abc29229e7862ada6891d1ccbe.svg
paper_authors.fullname: Zhu
paper_authors.type: user
paper_authors.user: YongmingZhu
paper_title: DreamActor-M1: Holistic, Expressive and Robust Human Image Animation
    with Hybrid Guidance
paper_summary: While recent image-based human animation methods achieve realistic body and
  facial motion synthesis, critical gaps remain in fine-grained holistic
  controllability, multi-scale adaptability, and long-term temporal coherence,
  which leads to their lower expressiveness and robustness. We propose a
  diffusion transformer (DiT) based framework, DreamActor-M1, with hybrid
  guidance to overcome these limitations. For motion guidance, our hybrid control
  signals that integrate implicit facial representations, 3D head spheres, and 3D
  body skeletons achieve robust control of facial expressions and body movements,
  while producing expressive and identity-preserving animations. For scale
  adaptation, to handle various body poses and image scales ranging from
  portraits to full-body views, we employ a progressive training strategy using
  data with varying resolutions and scales. For appearance guidance, we integrate
  motion patterns from sequential frames with complementary visual references,
  ensuring long-term temporal coherence for unseen regions during complex
  movements. Experiments demonstrate that our method outperforms the
  state-of-the-art works, delivering expressive results for portraits,
  upper-body, and full-body generation with robust long-term consistency. Project
  Page: https://grisoon.github.io/DreamActor-M1/.
paper_ai_summary: A diffusion transformer framework, DreamActor-M1, enhances image-based human animation through hybrid guidance, achieving robust control, multi-scale adaptability, and long-term temporal coherence.
paper_ai_keywords: diffusion transformer DreamActor-M1 hybrid guidance implicit facial representations 3D head spheres 3D body skeletons robust control body movements progressive training strategy varying resolutions sequential frames visual references long-term temporal coherence expressive animations identity-preserving animations