title: EasyAnimate: A High-Performance Long Video Generation Method based on
    Transformer Architecture
summary: This paper presents EasyAnimate, an advanced method for video generation that
  leverages the power of transformer architecture for high-performance outcomes.
  We have expanded the DiT framework originally designed for 2D image synthesis
  to accommodate the complexities of 3D video generation by incorporating a
  motion module block. It is used to capture temporal dynamics, thereby ensuring
  the production of consistent frames and seamless motion transitions. The motion
  module can be adapted to various DiT baseline methods to generate video with
  different styles. It can also generate videos with different frame rates and
  resolutions during both training and inference phases, suitable for both images
  and videos. Moreover, we introduce slice VAE, a novel approach to condense the
  temporal axis, facilitating the generation of long duration videos. Currently,
  EasyAnimate exhibits the proficiency to generate videos with 144 frames. We
  provide a holistic ecosystem for video production based on DiT, encompassing
  aspects such as data pre-processing, VAE training, DiT models training (both
  the baseline model and LoRA model), and end-to-end video inference. Code is
  available at: https://github.com/aigc-apps/EasyAnimate. We are continuously
  working to enhance the performance of our method.
paper_authors._id: 66588c57893215131946bea6
paper_authors.name: Jiaqi Xu
paper_authors._id: 66588c57893215131946bea7
paper_authors.name: Xinyi Zou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-31T10:50:39.399Z
paper_authors._id: 6419b3e98e0baaeed5a18333
paper_authors.avatarUrl: /avatars/915bc1d711630c78c0c2760ebdf104b7.svg
paper_authors.fullname: Xinyi Zou
paper_authors.type: user
paper_authors.user: Zouxinyi
paper_authors._id: 66588c57893215131946bea8
paper_authors.name: Kunzhe Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-31T10:50:45.742Z
paper_authors._id: 64c22cd08d3481788e212e9c
paper_authors.avatarUrl: /avatars/4484668c1090990ae678050bc07f9a68.svg
paper_authors.fullname: Kunzhe Huang
paper_authors.type: user
paper_authors.user: hkunzhe
paper_authors._id: 66588c57893215131946bea9
paper_authors.name: Yunkuo Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-31T10:50:27.607Z
paper_authors._id: 62009887577fcfa0ce82f1b0
paper_authors.avatarUrl: /avatars/d664ff2943148e48c228f3a1098f4264.svg
paper_authors.fullname: chenyunkuo
paper_authors.type: user
paper_authors.user: yunkchen
paper_authors._id: 66588c57893215131946beaa
paper_authors.name: Bo Liu
paper_authors._id: 66588c57893215131946beab
paper_authors.name: MengLi Cheng
paper_authors._id: 66588c57893215131946beac
paper_authors.name: Xing Shi
paper_authors._id: 66588c57893215131946bead
paper_authors.name: Jun Huang
paper_title: EasyAnimate: A High-Performance Long Video Generation Method based on
    Transformer Architecture
paper_summary: This paper presents EasyAnimate, an advanced method for video generation that
  leverages the power of transformer architecture for high-performance outcomes.
  We have expanded the DiT framework originally designed for 2D image synthesis
  to accommodate the complexities of 3D video generation by incorporating a
  motion module block. It is used to capture temporal dynamics, thereby ensuring
  the production of consistent frames and seamless motion transitions. The motion
  module can be adapted to various DiT baseline methods to generate video with
  different styles. It can also generate videos with different frame rates and
  resolutions during both training and inference phases, suitable for both images
  and videos. Moreover, we introduce slice VAE, a novel approach to condense the
  temporal axis, facilitating the generation of long duration videos. Currently,
  EasyAnimate exhibits the proficiency to generate videos with 144 frames. We
  provide a holistic ecosystem for video production based on DiT, encompassing
  aspects such as data pre-processing, VAE training, DiT models training (both
  the baseline model and LoRA model), and end-to-end video inference. Code is
  available at: https://github.com/aigc-apps/EasyAnimate. We are continuously
  working to enhance the performance of our method.
paper_ai_summary: EasyAnimate leverages transformer architecture to generate 3D videos with consistent motion and varying frame rates and resolutions using a motion module and slice VAE.
paper_ai_keywords: transformer architecture DiT framework motion module block temporal dynamics slice VAE video inference LoRA model