title: OSV: One Step is Enough for High-Quality Image to Video Generation
summary: Video diffusion models have shown great potential in generating high-quality
  videos, making them an increasingly popular focus. However, their inherent
  iterative nature leads to substantial computational and time costs. While
  efforts have been made to accelerate video diffusion by reducing inference
  steps (through techniques like consistency distillation) and GAN training
  (these approaches often fall short in either performance or training
  stability). In this work, we introduce a two-stage training framework that
  effectively combines consistency distillation with GAN training to address
  these challenges. Additionally, we propose a novel video discriminator design,
  which eliminates the need for decoding the video latents and improves the final
  performance. Our model is capable of producing high-quality videos in merely
  one-step, with the flexibility to perform multi-step refinement for further
  performance enhancement. Our quantitative evaluation on the OpenWebVid-1M
  benchmark shows that our model significantly outperforms existing methods.
  Notably, our 1-step performance(FVD 171.15) exceeds the 8-step performance of
  the consistency distillation based method, AnimateLCM (FVD 184.79), and
  approaches the 25-step performance of advanced Stable Video Diffusion (FVD
  156.94).
paper_authors._id: 66ea33e408671611fc90a2c6
paper_authors.name: Xiaofeng Mao
paper_authors._id: 66ea33e408671611fc90a2c7
paper_authors.name: Zhengkai Jiang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-01T13:10:43.706Z
paper_authors._id: 6371dda2ffc0489ed7d712c2
paper_authors.avatarUrl: /avatars/dcb80cae00535c22af99baf86cb70d93.svg
paper_authors.fullname: zkjiang
paper_authors.type: user
paper_authors.user: justin-zk
paper_authors._id: 66ea33e408671611fc90a2c8
paper_authors.name: Fu-Yun Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-18T07:01:39.124Z
paper_authors._id: 63e9e92f20c109718713f5eb
paper_authors.avatarUrl: /avatars/9ff312e854d803e1a2e9e685a21d12f8.svg
paper_authors.fullname: Fu-Yun Wang
paper_authors.type: user
paper_authors.user: wangfuyun
paper_authors._id: 66ea33e408671611fc90a2c9
paper_authors.name: Wenbing Zhu
paper_authors._id: 66ea33e408671611fc90a2ca
paper_authors.name: Jiangning Zhang
paper_authors._id: 66ea33e408671611fc90a2cb
paper_authors.name: Hao Chen
paper_authors._id: 66ea33e408671611fc90a2cc
paper_authors.name: Mingmin Chi
paper_authors._id: 66ea33e408671611fc90a2cd
paper_authors.name: Yabiao Wang
paper_title: OSV: One Step is Enough for High-Quality Image to Video Generation
paper_summary: Video diffusion models have shown great potential in generating high-quality
  videos, making them an increasingly popular focus. However, their inherent
  iterative nature leads to substantial computational and time costs. While
  efforts have been made to accelerate video diffusion by reducing inference
  steps (through techniques like consistency distillation) and GAN training
  (these approaches often fall short in either performance or training
  stability). In this work, we introduce a two-stage training framework that
  effectively combines consistency distillation with GAN training to address
  these challenges. Additionally, we propose a novel video discriminator design,
  which eliminates the need for decoding the video latents and improves the final
  performance. Our model is capable of producing high-quality videos in merely
  one-step, with the flexibility to perform multi-step refinement for further
  performance enhancement. Our quantitative evaluation on the OpenWebVid-1M
  benchmark shows that our model significantly outperforms existing methods.
  Notably, our 1-step performance(FVD 171.15) exceeds the 8-step performance of
  the consistency distillation based method, AnimateLCM (FVD 184.79), and
  approaches the 25-step performance of advanced Stable Video Diffusion (FVD
  156.94).
paper_ai_keywords: diffusion models consistency distillation GAN training video discriminator video latents multi-step refinement OpenWebVid-1M FVD (Fréchet Video Distance)