title: SF-V: Single Forward Video Generation Model
summary: Diffusion-based video generation models have demonstrated remarkable success
  in obtaining high-fidelity videos through the iterative denoising process.
  However, these models require multiple denoising steps during sampling,
  resulting in high computational costs. In this work, we propose a novel
  approach to obtain single-step video generation models by leveraging
  adversarial training to fine-tune pre-trained video diffusion models. We show
  that, through the adversarial training, the multi-steps video diffusion model,
  i.e., Stable Video Diffusion (SVD), can be trained to perform single forward
  pass to synthesize high-quality videos, capturing both temporal and spatial
  dependencies in the video data. Extensive experiments demonstrate that our
  method achieves competitive generation quality of synthesized videos with
  significantly reduced computational overhead for the denoising process (i.e.,
  around 23times speedup compared with SVD and 6times speedup compared with
  existing works, with even better generation quality), paving the way for
  real-time video synthesis and editing. More visualization results are made
  publicly available at https://snap-research.github.io/SF-V.
paper_authors._id: 666269c6fa2e0d45cd7c9414
paper_authors.name: Zhixing Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-07T09:20:12.160Z
paper_authors._id: 63081b822d33960e6b78ccf0
paper_authors.avatarUrl: /avatars/cbc3b007fdc050544a4bcc9881c63beb.svg
paper_authors.fullname: Zhixing Zhang
paper_authors.type: user
paper_authors.user: zhixingz
paper_authors._id: 666269c6fa2e0d45cd7c9415
paper_authors.name: Yanyu Li
paper_authors._id: 666269c6fa2e0d45cd7c9416
paper_authors.name: Yushu Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-07T13:30:29.235Z
paper_authors._id: 6566d82954c6bc01e54b65bf
paper_authors.avatarUrl: /avatars/bcaaa2bcdb6ba8d4042a07def99cfa08.svg
paper_authors.fullname: Yushu Wu
paper_authors.type: user
paper_authors.user: wuyushuwys
paper_authors._id: 666269c6fa2e0d45cd7c9417
paper_authors.name: Yanwu Xu
paper_authors._id: 666269c6fa2e0d45cd7c9418
paper_authors.name: Anil Kag
paper_authors._id: 666269c6fa2e0d45cd7c9419
paper_authors.name: Ivan Skorokhodov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-07T13:31:04.054Z
paper_authors._id: 63610db13c7147fae7de88e3
paper_authors.avatarUrl: /avatars/d7e97a16cfee39e1e50d7a5b747876f1.svg
paper_authors.fullname: Ivan Skorokhodov
paper_authors.type: user
paper_authors.user: universome
paper_authors._id: 666269c6fa2e0d45cd7c941a
paper_authors.name: Willi Menapace
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-07T13:31:09.708Z
paper_authors._id: 6315358a362e3e95ea538081
paper_authors.avatarUrl: /avatars/3b089a25a87c2e83c6b23ccb5d2dc73e.svg
paper_authors.fullname: Willi Menapace
paper_authors.type: user
paper_authors.user: willi-menapace
paper_authors._id: 666269c6fa2e0d45cd7c941b
paper_authors.name: Aliaksandr Siarohin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-07T13:31:16.337Z
paper_authors._id: 64276311eb9a0ed86180715b
paper_authors.avatarUrl: /avatars/76f933cd549f10e5e2db379de235d304.svg
paper_authors.fullname: Aliaksandr Siarohin
paper_authors.type: user
paper_authors.user: aliaksandr-siarohin
paper_authors._id: 666269c6fa2e0d45cd7c941c
paper_authors.name: Junli Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-07T13:31:22.924Z
paper_authors._id: 63f54aa73aa49d8cb97b84bc
paper_authors.avatarUrl: /avatars/c73c5870039611ab9162daad46a1ba20.svg
paper_authors.fullname: junli cao
paper_authors.type: user
paper_authors.user: jlcao2
paper_authors._id: 666269c6fa2e0d45cd7c941d
paper_authors.name: Dimitris Metaxas
paper_authors._id: 666269c6fa2e0d45cd7c941e
paper_authors.name: Sergey Tulyakov
paper_authors._id: 666269c6fa2e0d45cd7c941f
paper_authors.name: Jian Ren
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-07T09:20:03.858Z
paper_authors._id: 61f19829233c91cbd2f79e70
paper_authors.avatarUrl: /avatars/a0735a94542b4f7cda5aed8bc4be0538.svg
paper_authors.fullname: Jian Ren
paper_authors.type: user
paper_authors.user: alanspike
paper_title: SF-V: Single Forward Video Generation Model
paper_summary: Diffusion-based video generation models have demonstrated remarkable success
  in obtaining high-fidelity videos through the iterative denoising process.
  However, these models require multiple denoising steps during sampling,
  resulting in high computational costs. In this work, we propose a novel
  approach to obtain single-step video generation models by leveraging
  adversarial training to fine-tune pre-trained video diffusion models. We show
  that, through the adversarial training, the multi-steps video diffusion model,
  i.e., Stable Video Diffusion (SVD), can be trained to perform single forward
  pass to synthesize high-quality videos, capturing both temporal and spatial
  dependencies in the video data. Extensive experiments demonstrate that our
  method achieves competitive generation quality of synthesized videos with
  significantly reduced computational overhead for the denoising process (i.e.,
  around 23times speedup compared with SVD and 6times speedup compared with
  existing works, with even better generation quality), paving the way for
  real-time video synthesis and editing. More visualization results are made
  publicly available at https://snap-research.github.io/SF-V.
paper_ai_keywords: diffusion-based video generation models iterative denoising process computational costs adversarial training pre-trained video diffusion models Stable Video Diffusion (SVD) single forward pass high-quality videos temporal dependencies spatial dependencies video data computational overhead denoising process real-time video synthesis real-time video editing