title: How Animals Dance (When You're Not Looking)
summary: We present a keyframe-based framework for generating music-synchronized,
  choreography aware animal dance videos. Starting from a few keyframes
  representing distinct animal poses -- generated via text-to-image prompting or
  GPT-4o -- we formulate dance synthesis as a graph optimization problem: find
  the optimal keyframe structure that satisfies a specified choreography pattern
  of beats, which can be automatically estimated from a reference dance video. We
  also introduce an approach for mirrored pose image generation, essential for
  capturing symmetry in dance. In-between frames are synthesized using an video
  diffusion model. With as few as six input keyframes, our method can produce up
  to 30 second dance videos across a wide range of animals and music tracks.
paper_authors._id: 6839e1b656bcc85d9fc73669
paper_authors.name: Xiaojuan Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-02T07:44:02.906Z
paper_authors._id: 643ee9a6022a0350efd745d1
paper_authors.avatarUrl: /avatars/c8c17689ed88b1966d2a36361e3f83cc.svg
paper_authors.fullname: Xiaojuan Wang
paper_authors.type: user
paper_authors.user: xiaojwan
paper_authors._id: 6839e1b656bcc85d9fc7366a
paper_authors.name: Aleksander Holynski
paper_authors._id: 6839e1b656bcc85d9fc7366b
paper_authors.name: Brian Curless
paper_authors._id: 6839e1b656bcc85d9fc7366c
paper_authors.name: Ira Kemelmacher
paper_authors._id: 6839e1b656bcc85d9fc7366d
paper_authors.name: Steve Seitz
paper_title: How Animals Dance (When You're Not Looking)
paper_summary: We present a keyframe-based framework for generating music-synchronized,
  choreography aware animal dance videos. Starting from a few keyframes
  representing distinct animal poses -- generated via text-to-image prompting or
  GPT-4o -- we formulate dance synthesis as a graph optimization problem: find
  the optimal keyframe structure that satisfies a specified choreography pattern
  of beats, which can be automatically estimated from a reference dance video. We
  also introduce an approach for mirrored pose image generation, essential for
  capturing symmetry in dance. In-between frames are synthesized using an video
  diffusion model. With as few as six input keyframes, our method can produce up
  to 30 second dance videos across a wide range of animals and music tracks.
paper_ai_summary: A framework synthesizes music-synchronized animal dance videos from keyframes using graph optimization and video diffusion models, supporting symmetry and wide animal and music variation.
paper_ai_keywords: graph optimization video diffusion model mirrored pose image generation