title: Make Pixels Dance: High-Dynamic Video Generation
summary: Creating high-dynamic videos such as motion-rich actions and sophisticated
  visual effects poses a significant challenge in the field of artificial
  intelligence. Unfortunately, current state-of-the-art video generation methods,
  primarily focusing on text-to-video generation, tend to produce video clips
  with minimal motions despite maintaining high fidelity. We argue that relying
  solely on text instructions is insufficient and suboptimal for video
  generation. In this paper, we introduce PixelDance, a novel approach based on
  diffusion models that incorporates image instructions for both the first and
  last frames in conjunction with text instructions for video generation.
  Comprehensive experimental results demonstrate that PixelDance trained with
  public data exhibits significantly better proficiency in synthesizing videos
  with complex scenes and intricate motions, setting a new standard for video
  generation.
paper_authors._id: 655c34faf63eaf2a751b2bda
paper_authors.name: Yan Zeng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-21T10:27:02.859Z
paper_authors._id: 640ef2b2f2d7c41a1e9b6013
paper_authors.avatarUrl: /avatars/556263c07df93942c22cda64e65b2e41.svg
paper_authors.fullname: Yan Zeng
paper_authors.type: user
paper_authors.user: zengyan-97
paper_authors._id: 655c34faf63eaf2a751b2bdb
paper_authors.name: Guoqiang Wei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-21T10:27:09.416Z
paper_authors._id: 63160a1d29411a6864b0e585
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1668083049513-63160a1d29411a6864b0e585.jpeg
paper_authors.fullname: Guoqiang Wei
paper_authors.type: user
paper_authors.user: weigq
paper_authors._id: 655c34faf63eaf2a751b2bdc
paper_authors.name: Jiani Zheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-11-21T15:00:50.619Z
paper_authors._id: 64531f631a57e1179c203e6b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64531f631a57e1179c203e6b/C_J7pXFLqoJoHYPPhK3J9.jpeg
paper_authors.fullname: zjn
paper_authors.type: user
paper_authors.user: garlicisnotmyfavor
paper_authors._id: 655c34faf63eaf2a751b2bdd
paper_authors.name: Jiaxin Zou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-11-21T13:04:54.854Z
paper_authors._id: 655c485e71ed242b42701fb0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/655c485e71ed242b42701fb0/7d6lHB-wMy7xNj3dWIp5y.jpeg
paper_authors.fullname: Jiaxin Zou
paper_authors.type: user
paper_authors.user: shayeboshi
paper_authors._id: 655c34faf63eaf2a751b2bde
paper_authors.name: Yang Wei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-21T10:47:43.045Z
paper_authors._id: 631a089f223d36c5706c7ba3
paper_authors.avatarUrl: /avatars/a2b8b303d5d912b531130be44b82f601.svg
paper_authors.fullname: Yang Wei
paper_authors.type: user
paper_authors.user: godweiyang
paper_authors._id: 655c34faf63eaf2a751b2bdf
paper_authors.name: Yuchen Zhang
paper_authors._id: 655c34faf63eaf2a751b2be0
paper_authors.name: Hang Li
paper_title: Make Pixels Dance: High-Dynamic Video Generation
paper_summary: Creating high-dynamic videos such as motion-rich actions and sophisticated
  visual effects poses a significant challenge in the field of artificial
  intelligence. Unfortunately, current state-of-the-art video generation methods,
  primarily focusing on text-to-video generation, tend to produce video clips
  with minimal motions despite maintaining high fidelity. We argue that relying
  solely on text instructions is insufficient and suboptimal for video
  generation. In this paper, we introduce PixelDance, a novel approach based on
  diffusion models that incorporates image instructions for both the first and
  last frames in conjunction with text instructions for video generation.
  Comprehensive experimental results demonstrate that PixelDance trained with
  public data exhibits significantly better proficiency in synthesizing videos
  with complex scenes and intricate motions, setting a new standard for video
  generation.
paper_ai_summary: PixelDance, a diffusion model-based approach, generates high-dynamic videos by incorporating image instructions for first and last frames alongside text instructions, surpassing current text-to-video methods in complexity and motion.
paper_ai_keywords: diffusion models image instructions text instructions video generation complex scenes intricate motions