title: EchoVideo: Identity-Preserving Human Video Generation by Multimodal
    Feature Fusion
summary: Recent advancements in video generation have significantly impacted various
  downstream applications, particularly in identity-preserving video generation
  (IPT2V). However, existing methods struggle with "copy-paste" artifacts and low
  similarity issues, primarily due to their reliance on low-level facial image
  information. This dependence can result in rigid facial appearances and
  artifacts reflecting irrelevant details. To address these challenges, we
  propose EchoVideo, which employs two key strategies: (1) an Identity Image-Text
  Fusion Module (IITF) that integrates high-level semantic features from text,
  capturing clean facial identity representations while discarding occlusions,
  poses, and lighting variations to avoid the introduction of artifacts; (2) a
  two-stage training strategy, incorporating a stochastic method in the second
  phase to randomly utilize shallow facial information. The objective is to
  balance the enhancements in fidelity provided by shallow features while
  mitigating excessive reliance on them. This strategy encourages the model to
  utilize high-level features during training, ultimately fostering a more robust
  representation of facial identities. EchoVideo effectively preserves facial
  identities and maintains full-body integrity. Extensive experiments demonstrate
  that it achieves excellent results in generating high-quality, controllability
  and fidelity videos.
paper_authors._id: 6793480ec6fd669f7341cf41
paper_authors.name: Jiangchuan Wei
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-02-14T14:24:45.099Z
paper_authors._id: 67af4daa602736dd990c1d43
paper_authors.avatarUrl: /avatars/802d512c20ca9a9c6ffa49aa5f98b96c.svg
paper_authors.fullname: weijiangchuan
paper_authors.type: user
paper_authors.user: weijiangchuan
paper_authors._id: 6793480ec6fd669f7341cf42
paper_authors.name: Shiyue Yan
paper_authors._id: 6793480ec6fd669f7341cf43
paper_authors.name: Wenfeng Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-24T10:18:06.509Z
paper_authors._id: 6676c4f86f2ac48ee6c2f4d4
paper_authors.avatarUrl: /avatars/fea4e5be4da7a7047df567a4aa86de0c.svg
paper_authors.fullname: linwenfeng
paper_authors.type: user
paper_authors.user: linwf
paper_authors._id: 6793480ec6fd669f7341cf44
paper_authors.name: Boyuan Liu
paper_authors._id: 6793480ec6fd669f7341cf45
paper_authors.name: Renjie Chen
paper_authors._id: 6793480ec6fd669f7341cf46
paper_authors.name: Mingyu Guo
paper_title: EchoVideo: Identity-Preserving Human Video Generation by Multimodal
    Feature Fusion
paper_summary: Recent advancements in video generation have significantly impacted various
  downstream applications, particularly in identity-preserving video generation
  (IPT2V). However, existing methods struggle with "copy-paste" artifacts and low
  similarity issues, primarily due to their reliance on low-level facial image
  information. This dependence can result in rigid facial appearances and
  artifacts reflecting irrelevant details. To address these challenges, we
  propose EchoVideo, which employs two key strategies: (1) an Identity Image-Text
  Fusion Module (IITF) that integrates high-level semantic features from text,
  capturing clean facial identity representations while discarding occlusions,
  poses, and lighting variations to avoid the introduction of artifacts; (2) a
  two-stage training strategy, incorporating a stochastic method in the second
  phase to randomly utilize shallow facial information. The objective is to
  balance the enhancements in fidelity provided by shallow features while
  mitigating excessive reliance on them. This strategy encourages the model to
  utilize high-level features during training, ultimately fostering a more robust
  representation of facial identities. EchoVideo effectively preserves facial
  identities and maintains full-body integrity. Extensive experiments demonstrate
  that it achieves excellent results in generating high-quality, controllability
  and fidelity videos.
paper_ai_keywords: Identity Image-Text Fusion Module (IITF) high-level deterministic features facial identity representations stochastic method shallow facial information full-body integrity high-quality, controllability and fidelity videos