title: ExVideo: Extending Video Diffusion Models via Parameter-Efficient
    Post-Tuning
summary: Recently, advancements in video synthesis have attracted significant
  attention. Video synthesis models such as AnimateDiff and Stable Video
  Diffusion have demonstrated the practical applicability of diffusion models in
  creating dynamic visual content. The emergence of SORA has further spotlighted
  the potential of video generation technologies. Nonetheless, the extension of
  video lengths has been constrained by the limitations in computational
  resources. Most existing video synthesis models can only generate short video
  clips. In this paper, we propose a novel post-tuning methodology for video
  synthesis models, called ExVideo. This approach is designed to enhance the
  capability of current video synthesis models, allowing them to produce content
  over extended temporal durations while incurring lower training expenditures.
  In particular, we design extension strategies across common temporal model
  architectures respectively, including 3D convolution, temporal attention, and
  positional embedding. To evaluate the efficacy of our proposed post-tuning
  approach, we conduct extension training on the Stable Video Diffusion model.
  Our approach augments the model's capacity to generate up to 5times its
  original number of frames, requiring only 1.5k GPU hours of training on a
  dataset comprising 40k videos. Importantly, the substantial increase in video
  length doesn't compromise the model's innate generalization capabilities, and
  the model showcases its advantages in generating videos of diverse styles and
  resolutions. We will release the source code and the enhanced model publicly.
paper_authors._id: 6674f66fbd6e32596f69e9c8
paper_authors.name: Zhongjie Duan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-21T12:08:36.125Z
paper_authors._id: 6364bec8ad2c9e8b672776e6
paper_authors.avatarUrl: /avatars/05293b4047437480da22f4622236d992.svg
paper_authors.fullname: Zhongjie Duan
paper_authors.type: user
paper_authors.user: Artiprocher
paper_authors._id: 6674f66fbd6e32596f69e9c9
paper_authors.name: Wenmeng Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-21T12:08:42.211Z
paper_authors._id: 623c6253389748c9f72ca287
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1654828369523-623c6253389748c9f72ca287.jpeg
paper_authors.fullname: wenmeng zhou
paper_authors.type: user
paper_authors.user: wenmengzhou
paper_authors._id: 6674f66fbd6e32596f69e9ca
paper_authors.name: Cen Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-21T12:09:24.261Z
paper_authors._id: 661e91481f06f858880080aa
paper_authors.avatarUrl: /avatars/3e2a714106db5b90b6f99b0cc3b984c2.svg
paper_authors.fullname: chencen
paper_authors.type: user
paper_authors.user: clh13600
paper_authors._id: 6674f66fbd6e32596f69e9cb
paper_authors.name: Yaliang Li
paper_authors._id: 6674f66fbd6e32596f69e9cc
paper_authors.name: Weining Qian
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-21T12:09:06.181Z
paper_authors._id: 643916835aa69077ffb27c8a
paper_authors.avatarUrl: /avatars/3eeb0b8ea703d3c873839ad0fd062352.svg
paper_authors.fullname: Weining Qian
paper_authors.type: user
paper_authors.user: wnqian
paper_title: ExVideo: Extending Video Diffusion Models via Parameter-Efficient
    Post-Tuning
paper_summary: Recently, advancements in video synthesis have attracted significant
  attention. Video synthesis models such as AnimateDiff and Stable Video
  Diffusion have demonstrated the practical applicability of diffusion models in
  creating dynamic visual content. The emergence of SORA has further spotlighted
  the potential of video generation technologies. Nonetheless, the extension of
  video lengths has been constrained by the limitations in computational
  resources. Most existing video synthesis models can only generate short video
  clips. In this paper, we propose a novel post-tuning methodology for video
  synthesis models, called ExVideo. This approach is designed to enhance the
  capability of current video synthesis models, allowing them to produce content
  over extended temporal durations while incurring lower training expenditures.
  In particular, we design extension strategies across common temporal model
  architectures respectively, including 3D convolution, temporal attention, and
  positional embedding. To evaluate the efficacy of our proposed post-tuning
  approach, we conduct extension training on the Stable Video Diffusion model.
  Our approach augments the model's capacity to generate up to 5times its
  original number of frames, requiring only 1.5k GPU hours of training on a
  dataset comprising 40k videos. Importantly, the substantial increase in video
  length doesn't compromise the model's innate generalization capabilities, and
  the model showcases its advantages in generating videos of diverse styles and
  resolutions. We will release the source code and the enhanced model publicly.
paper_ai_keywords: diffusion models AnimateDiff Stable Video Diffusion SORA video synthesis modular post-tuning methodology ExVideo 3D convolution temporal attention positional embedding extension training generalization capabilities