title: MagicInfinite: Generating Infinite Talking Videos with Your Words and
    Voice
summary: We present MagicInfinite, a novel diffusion Transformer (DiT) framework that
  overcomes traditional portrait animation limitations, delivering high-fidelity
  results across diverse character types-realistic humans, full-body figures, and
  stylized anime characters. It supports varied facial poses, including
  back-facing views, and animates single or multiple characters with input masks
  for precise speaker designation in multi-character scenes. Our approach tackles
  key challenges with three innovations: (1) 3D full-attention mechanisms with a
  sliding window denoising strategy, enabling infinite video generation with
  temporal coherence and visual quality across diverse character styles; (2) a
  two-stage curriculum learning scheme, integrating audio for lip sync, text for
  expressive dynamics, and reference images for identity preservation, enabling
  flexible multi-modal control over long sequences; and (3) region-specific masks
  with adaptive loss functions to balance global textual control and local audio
  guidance, supporting speaker-specific animations. Efficiency is enhanced via
  our innovative unified step and cfg distillation techniques, achieving a 20x
  inference speed boost over the basemodel: generating a 10 second 540x540p video
  in 10 seconds or 720x720p in 30 seconds on 8 H100 GPUs, without quality loss.
  Evaluations on our new benchmark demonstrate MagicInfinite's superiority in
  audio-lip synchronization, identity preservation, and motion naturalness across
  diverse scenarios. It is publicly available at https://www.hedra.com/, with
  examples at https://magicinfinite.github.io/.
paper_authors._id: 67d129d732b4bbfb938321a1
paper_authors.name: Hongwei Yi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:35:50.514Z
paper_authors._id: 67cda39ea1aad70c919a900f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/vyF20L4ng460PaAIDeO97.png
paper_authors.fullname: hwy
paper_authors.type: user
paper_authors.user: hongweiyi
paper_authors._id: 67d129d732b4bbfb938321a2
paper_authors.name: Tian Ye
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-12T08:50:12.319Z
paper_authors._id: 66015e8aa4d296af07de538e
paper_authors.avatarUrl: /avatars/a1295c631cc2646282c545859975ce4c.svg
paper_authors.fullname: Ye
paper_authors.type: user
paper_authors.user: Owen777
paper_authors._id: 67d129d732b4bbfb938321a3
paper_authors.name: Shitong Shao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-19T09:46:40.444Z
paper_authors._id: 64aa48245fc663fee24125be
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64aa48245fc663fee24125be/8Me4rbY3XdY48p9QyEpaA.jpeg
paper_authors.fullname: ShanShan
paper_authors.type: user
paper_authors.user: sst12345
paper_authors._id: 67d129d732b4bbfb938321a4
paper_authors.name: Xuancheng Yang
paper_authors._id: 67d129d732b4bbfb938321a5
paper_authors.name: Jiantong Zhao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:36:27.599Z
paper_authors._id: 655359eee0a82177ae5faf50
paper_authors.avatarUrl: /avatars/2792e0349eaf32d1bb52a2bae23f143f.svg
paper_authors.fullname: Jiantong Zhao
paper_authors.type: user
paper_authors.user: zjt000125
paper_authors._id: 67d129d732b4bbfb938321a6
paper_authors.name: Hanzhong Guo
paper_authors._id: 67d129d732b4bbfb938321a7
paper_authors.name: Terrance Wang
paper_authors._id: 67d129d732b4bbfb938321a8
paper_authors.name: Qingyu Yin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:36:56.627Z
paper_authors._id: 6453cb22908e259483c0a061
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6453cb22908e259483c0a061/hMgdwZUsUbgquGalzPGzV.jpeg
paper_authors.fullname: Qingyu_Yin
paper_authors.type: user
paper_authors.user: MikaStars39
paper_authors._id: 67d129d732b4bbfb938321a9
paper_authors.name: Zeke Xie
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-16T21:17:48.716Z
paper_authors._id: 67a5fb17baac478d94c59fec
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/gMtdNiD8ztyAjB_weexcK.png
paper_authors.fullname: Z.X.
paper_authors.type: user
paper_authors.user: xLeaF
paper_authors._id: 67d129d732b4bbfb938321aa
paper_authors.name: Lei Zhu
paper_authors._id: 67d129d732b4bbfb938321ab
paper_authors.name: Wei Li
paper_authors._id: 67d129d732b4bbfb938321ac
paper_authors.name: Michael Lingelbach
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:37:38.347Z
paper_authors._id: 6170a328dff0ef663e421ac0
paper_authors.avatarUrl: /avatars/d1699613947cc147ea087d41608d6f46.svg
paper_authors.fullname: Michael Lingelbach
paper_authors.type: user
paper_authors.user: mjlbach
paper_authors._id: 67d129d732b4bbfb938321ad
paper_authors.name: Daquan Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:37:21.244Z
paper_authors._id: 63fe0b160c1bbe8e29d2dd32
paper_authors.avatarUrl: /avatars/bc574036287170a77057893efaa48e2d.svg
paper_authors.fullname: Zhou
paper_authors.type: user
paper_authors.user: DaQuan21
paper_title: MagicInfinite: Generating Infinite Talking Videos with Your Words and
    Voice
paper_summary: We present MagicInfinite, a novel diffusion Transformer (DiT) framework that
  overcomes traditional portrait animation limitations, delivering high-fidelity
  results across diverse character types-realistic humans, full-body figures, and
  stylized anime characters. It supports varied facial poses, including
  back-facing views, and animates single or multiple characters with input masks
  for precise speaker designation in multi-character scenes. Our approach tackles
  key challenges with three innovations: (1) 3D full-attention mechanisms with a
  sliding window denoising strategy, enabling infinite video generation with
  temporal coherence and visual quality across diverse character styles; (2) a
  two-stage curriculum learning scheme, integrating audio for lip sync, text for
  expressive dynamics, and reference images for identity preservation, enabling
  flexible multi-modal control over long sequences; and (3) region-specific masks
  with adaptive loss functions to balance global textual control and local audio
  guidance, supporting speaker-specific animations. Efficiency is enhanced via
  our innovative unified step and cfg distillation techniques, achieving a 20x
  inference speed boost over the basemodel: generating a 10 second 540x540p video
  in 10 seconds or 720x720p in 30 seconds on 8 H100 GPUs, without quality loss.
  Evaluations on our new benchmark demonstrate MagicInfinite's superiority in
  audio-lip synchronization, identity preservation, and motion naturalness across
  diverse scenarios. It is publicly available at https://www.hedra.com/, with
  examples at https://magicinfinite.github.io/.
paper_ai_summary: MagicInfinite, a diffusion Transformer framework, provides high-fidelity portrait animations across various styles with techniques like 3D full-attention, curriculum learning, region-specific masks, and step and cfg distillation.
paper_ai_keywords: diffusion Transformer DiT 3D full-attention mechanisms sliding window denoising infinite video generation temporal coherence visual quality two-stage curriculum learning audio-lip synchronization expressive dynamics identity preservation flexible multi-modal control region-specific masks adaptive loss functions unified step and cfg distillation inference speed boost benchmark motion naturalness