title: DisCo: Disentangled Control for Referring Human Dance Generation in Real
    World
summary: Generative AI has made significant strides in computer vision, particularly
  in image/video synthesis conditioned on text descriptions. Despite the
  advancements, it remains challenging especially in the generation of
  human-centric content such as dance synthesis. Existing dance synthesis methods
  struggle with the gap between synthesized content and real-world dance
  scenarios. In this paper, we define a new problem setting: Referring Human
  Dance Generation, which focuses on real-world dance scenarios with three
  important properties: (i) Faithfulness: the synthesis should retain the
  appearance of both human subject foreground and background from the reference
  image, and precisely follow the target pose; (ii) Generalizability: the model
  should generalize to unseen human subjects, backgrounds, and poses; (iii)
  Compositionality: it should allow for composition of seen/unseen subjects,
  backgrounds, and poses from different sources. To address these challenges, we
  introduce a novel approach, DISCO, which includes a novel model architecture
  with disentangled control to improve the faithfulness and compositionality of
  dance synthesis, and an effective human attribute pre-training for better
  generalizability to unseen humans. Extensive qualitative and quantitative
  results demonstrate that DISCO can generate high-quality human dance images and
  videos with diverse appearances and flexible motions. Code, demo, video and
  visualization are available at: https://disco-dance.github.io/.
paper_authors._id: 64a39ecbc9b4db7cc77e31c0
paper_authors.name: Tan Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-04T19:16:38.476Z
paper_authors._id: 62a9cb5c0036766ec1b67f09
paper_authors.avatarUrl: /avatars/5a9c5b41ef3eebbe7b21184b289ade68.svg
paper_authors.fullname: Tan Wang
paper_authors.type: user
paper_authors.user: ReaLMe
paper_authors._id: 64a39ecbc9b4db7cc77e31c1
paper_authors.name: Linjie Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-08-08T01:20:23.803Z
paper_authors._id: 63db16fff03c3d71ef397206
paper_authors.avatarUrl: /avatars/bfb7e0d730b7d03302799d5d2828d97d.svg
paper_authors.fullname: Linjie Li
paper_authors.type: user
paper_authors.user: linjieli222
paper_authors._id: 64a39ecbc9b4db7cc77e31c2
paper_authors.name: Kevin Lin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-05T18:56:05.428Z
paper_authors._id: 6298fd95b58e71e2ac9f3ad8
paper_authors.avatarUrl: /avatars/7d34644d537bc5c17cf1e4ce4095355c.svg
paper_authors.fullname: Kevin Lin
paper_authors.type: user
paper_authors.user: kevinlin311tw
paper_authors._id: 64a39ecbc9b4db7cc77e31c3
paper_authors.name: Chung-Ching Lin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-13T09:33:59.307Z
paper_authors._id: 6669f406d21cf7d39b1d98ba
paper_authors.avatarUrl: /avatars/5ec270cb8bcb68786379d8a8bc411aaa.svg
paper_authors.fullname: CC Lin
paper_authors.type: user
paper_authors.user: cclin10
paper_authors._id: 64a39ecbc9b4db7cc77e31c4
paper_authors.name: Zhengyuan Yang
paper_authors._id: 64a39ecbc9b4db7cc77e31c5
paper_authors.name: Hanwang Zhang
paper_authors._id: 64a39ecbc9b4db7cc77e31c6
paper_authors.name: Zicheng Liu
paper_authors._id: 64a39ecbc9b4db7cc77e31c7
paper_authors.name: Lijuan Wang
paper_title: DisCo: Disentangled Control for Referring Human Dance Generation in Real
    World
paper_summary: Generative AI has made significant strides in computer vision, particularly
  in image/video synthesis conditioned on text descriptions. Despite the
  advancements, it remains challenging especially in the generation of
  human-centric content such as dance synthesis. Existing dance synthesis methods
  struggle with the gap between synthesized content and real-world dance
  scenarios. In this paper, we define a new problem setting: Referring Human
  Dance Generation, which focuses on real-world dance scenarios with three
  important properties: (i) Faithfulness: the synthesis should retain the
  appearance of both human subject foreground and background from the reference
  image, and precisely follow the target pose; (ii) Generalizability: the model
  should generalize to unseen human subjects, backgrounds, and poses; (iii)
  Compositionality: it should allow for composition of seen/unseen subjects,
  backgrounds, and poses from different sources. To address these challenges, we
  introduce a novel approach, DISCO, which includes a novel model architecture
  with disentangled control to improve the faithfulness and compositionality of
  dance synthesis, and an effective human attribute pre-training for better
  generalizability to unseen humans. Extensive qualitative and quantitative
  results demonstrate that DISCO can generate high-quality human dance images and
  videos with diverse appearances and flexible motions. Code, demo, video and
  visualization are available at: https://disco-dance.github.io/.
paper_ai_keywords: Referring Human Dance Generation disentangled control Dance synthesis Generalizability Compositionality DISCO human attribute pre-training high-quality human dance images flexible motions