title: DanceTogether! Identity-Preserving Multi-Person Interactive Video
    Generation
summary: Controllable video generation (CVG) has advanced rapidly, yet current systems
  falter when more than one actor must move, interact, and exchange positions
  under noisy control signals. We address this gap with DanceTogether, the first
  end-to-end diffusion framework that turns a single reference image plus
  independent pose-mask streams into long, photorealistic videos while strictly
  preserving every identity. A novel MaskPoseAdapter binds "who" and "how" at
  every denoising step by fusing robust tracking masks with semantically rich-but
  noisy-pose heat-maps, eliminating the identity drift and appearance bleeding
  that plague frame-wise pipelines. To train and evaluate at scale, we introduce
  (i) PairFS-4K, 26 hours of dual-skater footage with 7,000+ distinct IDs, (ii)
  HumanRob-300, a one-hour humanoid-robot interaction set for rapid cross-domain
  transfer, and (iii) TogetherVideoBench, a three-track benchmark centered on the
  DanceTogEval-100 test suite covering dance, boxing, wrestling, yoga, and figure
  skating. On TogetherVideoBench, DanceTogether outperforms the prior arts by a
  significant margin. Moreover, we show that a one-hour fine-tune yields
  convincing human-robot videos, underscoring broad generalization to embodied-AI
  and HRI tasks. Extensive ablations confirm that persistent identity-action
  binding is critical to these gains. Together, our model, datasets, and
  benchmark lift CVG from single-subject choreography to compositionally
  controllable, multi-actor interaction, opening new avenues for digital
  production, simulation, and embodied intelligence. Our video demos and code are
  available at https://DanceTog.github.io/.
paper_authors._id: 683463235a49ee2881ade255
paper_authors.name: Junhao Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-26T13:18:31.745Z
paper_authors._id: 652b80d4703b3743c25c87d5
paper_authors.avatarUrl: /avatars/3d9c0429fc6082cfb28e2dcd05845733.svg
paper_authors.fullname: Dream How Chen
paper_authors.type: user
paper_authors.user: yisuanwang
paper_authors._id: 683463235a49ee2881ade256
paper_authors.name: Mingjin Chen
paper_authors._id: 683463235a49ee2881ade257
paper_authors.name: Jianjin Xu
paper_authors._id: 683463235a49ee2881ade258
paper_authors.name: Xiang Li
paper_authors._id: 683463235a49ee2881ade259
paper_authors.name: Junting Dong
paper_authors._id: 683463235a49ee2881ade25a
paper_authors.name: Mingze Sun
paper_authors._id: 683463235a49ee2881ade25b
paper_authors.name: Puhua Jiang
paper_authors._id: 683463235a49ee2881ade25c
paper_authors.name: Hongxiang Li
paper_authors._id: 683463235a49ee2881ade25d
paper_authors.name: Yuhang Yang
paper_authors._id: 683463235a49ee2881ade25e
paper_authors.name: Hao Zhao
paper_authors._id: 683463235a49ee2881ade25f
paper_authors.name: Xiaoxiao Long
paper_authors._id: 683463235a49ee2881ade260
paper_authors.name: Ruqi Huang
paper_title: DanceTogether! Identity-Preserving Multi-Person Interactive Video
    Generation
paper_summary: Controllable video generation (CVG) has advanced rapidly, yet current systems
  falter when more than one actor must move, interact, and exchange positions
  under noisy control signals. We address this gap with DanceTogether, the first
  end-to-end diffusion framework that turns a single reference image plus
  independent pose-mask streams into long, photorealistic videos while strictly
  preserving every identity. A novel MaskPoseAdapter binds "who" and "how" at
  every denoising step by fusing robust tracking masks with semantically rich-but
  noisy-pose heat-maps, eliminating the identity drift and appearance bleeding
  that plague frame-wise pipelines. To train and evaluate at scale, we introduce
  (i) PairFS-4K, 26 hours of dual-skater footage with 7,000+ distinct IDs, (ii)
  HumanRob-300, a one-hour humanoid-robot interaction set for rapid cross-domain
  transfer, and (iii) TogetherVideoBench, a three-track benchmark centered on the
  DanceTogEval-100 test suite covering dance, boxing, wrestling, yoga, and figure
  skating. On TogetherVideoBench, DanceTogether outperforms the prior arts by a
  significant margin. Moreover, we show that a one-hour fine-tune yields
  convincing human-robot videos, underscoring broad generalization to embodied-AI
  and HRI tasks. Extensive ablations confirm that persistent identity-action
  binding is critical to these gains. Together, our model, datasets, and
  benchmark lift CVG from single-subject choreography to compositionally
  controllable, multi-actor interaction, opening new avenues for digital
  production, simulation, and embodied intelligence. Our video demos and code are
  available at https://DanceTog.github.io/.
paper_ai_summary: DanceTogether, an end-to-end diffusion framework, generates long, photorealistic multi-actor interaction videos from single reference images and pose-mask streams, outperforming existing systems.
paper_ai_keywords: diffusion framework MaskPoseAdapter denoising identity drift appearance bleeding PairFS-4K HumanRob-300 TogetherVideoBench DanceTogEval-100 embodied-AI HRI tasks persistent identity-action binding