title: DAWN: Dynamic Frame Avatar with Non-autoregressive Diffusion Framework
    for Talking Head Video Generation
summary: Talking head generation intends to produce vivid and realistic talking head
  videos from a single portrait and speech audio clip. Although significant
  progress has been made in diffusion-based talking head generation, almost all
  methods rely on autoregressive strategies, which suffer from limited context
  utilization beyond the current generation step, error accumulation, and slower
  generation speed. To address these challenges, we present DAWN (Dynamic frame
  Avatar With Non-autoregressive diffusion), a framework that enables all-at-once
  generation of dynamic-length video sequences. Specifically, it consists of two
  main components: (1) audio-driven holistic facial dynamics generation in the
  latent motion space, and (2) audio-driven head pose and blink generation.
  Extensive experiments demonstrate that our method generates authentic and vivid
  videos with precise lip motions, and natural pose/blink movements.
  Additionally, with a high generation speed, DAWN possesses strong extrapolation
  capabilities, ensuring the stable production of high-quality long videos. These
  results highlight the considerable promise and potential impact of DAWN in the
  field of talking head video generation. Furthermore, we hope that DAWN sparks
  further exploration of non-autoregressive approaches in diffusion models. Our
  code will be publicly at https://github.com/Hanbo-Cheng/DAWN-pytorch.
paper_authors._id: 67147d177b711cff63c0d610
paper_authors.name: Hanbo Cheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-21T07:47:17.268Z
paper_authors._id: 67147c30476f2449e228cad1
paper_authors.avatarUrl: /avatars/13976ebe52634c551d41b38da801e35d.svg
paper_authors.fullname: Hanbo-Cheng
paper_authors.type: user
paper_authors.user: Hanbo-Cheng
paper_authors._id: 67147d177b711cff63c0d611
paper_authors.name: Limin Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-21T12:15:07.918Z
paper_authors._id: 6716128fcf2b9210e5cab2db
paper_authors.avatarUrl: /avatars/254fa3b1409b844b38e4fd738f638fa6.svg
paper_authors.fullname: Limin-Lin
paper_authors.type: user
paper_authors.user: Limin-Lin
paper_authors._id: 67147d177b711cff63c0d612
paper_authors.name: Chenyu Liu
paper_authors._id: 67147d177b711cff63c0d613
paper_authors.name: Pengcheng Xia
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-21T12:15:13.738Z
paper_authors._id: 64946adc02a46148e275735c
paper_authors.avatarUrl: /avatars/26a20b321f495aa7af1481ba2af392bd.svg
paper_authors.fullname: Pengcheng Xia
paper_authors.type: user
paper_authors.user: page-xia
paper_authors._id: 67147d177b711cff63c0d614
paper_authors.name: Pengfei Hu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-21T12:15:19.747Z
paper_authors._id: 64291be28136224fee045d36
paper_authors.avatarUrl: /avatars/c0d155c053742478e8c6d6bad9e0bd4f.svg
paper_authors.fullname: Pengfei Hu
paper_authors.type: user
paper_authors.user: Bazhu
paper_authors._id: 67147d177b711cff63c0d615
paper_authors.name: Jiefeng Ma
paper_authors._id: 67147d177b711cff63c0d616
paper_authors.name: Jun Du
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-21T12:15:40.258Z
paper_authors._id: 5e0d54a0fcf41d740b699660
paper_authors.avatarUrl: /avatars/67dbb87baf7d75a4ec253cc706f819ce.svg
paper_authors.fullname: Tim Du
paper_authors.type: user
paper_authors.user: dujun
paper_authors._id: 67147d177b711cff63c0d617
paper_authors.name: Jia Pan
paper_title: DAWN: Dynamic Frame Avatar with Non-autoregressive Diffusion Framework
    for Talking Head Video Generation
paper_summary: Talking head generation intends to produce vivid and realistic talking head
  videos from a single portrait and speech audio clip. Although significant
  progress has been made in diffusion-based talking head generation, almost all
  methods rely on autoregressive strategies, which suffer from limited context
  utilization beyond the current generation step, error accumulation, and slower
  generation speed. To address these challenges, we present DAWN (Dynamic frame
  Avatar With Non-autoregressive diffusion), a framework that enables all-at-once
  generation of dynamic-length video sequences. Specifically, it consists of two
  main components: (1) audio-driven holistic facial dynamics generation in the
  latent motion space, and (2) audio-driven head pose and blink generation.
  Extensive experiments demonstrate that our method generates authentic and vivid
  videos with precise lip motions, and natural pose/blink movements.
  Additionally, with a high generation speed, DAWN possesses strong extrapolation
  capabilities, ensuring the stable production of high-quality long videos. These
  results highlight the considerable promise and potential impact of DAWN in the
  field of talking head video generation. Furthermore, we hope that DAWN sparks
  further exploration of non-autoregressive approaches in diffusion models. Our
  code will be publicly at https://github.com/Hanbo-Cheng/DAWN-pytorch.
paper_ai_summary: DAWN, a non-autoregressive diffusion framework, generates high-quality, realistic talking head videos with precise lip and natural head movement at high speed.
paper_ai_keywords: diffusion-based talking head generation autoregressive strategies latent motion space non-autoregressive diffusion audio-driven holistic facial dynamics audio-driven head pose audio-driven blink generation long videos high-quality videos precise lip motions natural pose/blink movements extrapolation capabilities