title: ViBiDSampler: Enhancing Video Interpolation Using Bidirectional
    Diffusion Sampler
summary: Recent progress in large-scale text-to-video (T2V) and image-to-video (I2V)
  diffusion models has greatly enhanced video generation, especially in terms of
  keyframe interpolation. However, current image-to-video diffusion models, while
  powerful in generating videos from a single conditioning frame, need adaptation
  for two-frame (start & end) conditioned generation, which is essential for
  effective bounded interpolation. Unfortunately, existing approaches that fuse
  temporally forward and backward paths in parallel often suffer from
  off-manifold issues, leading to artifacts or requiring multiple iterative
  re-noising steps. In this work, we introduce a novel, bidirectional sampling
  strategy to address these off-manifold issues without requiring extensive
  re-noising or fine-tuning. Our method employs sequential sampling along both
  forward and backward paths, conditioned on the start and end frames,
  respectively, ensuring more coherent and on-manifold generation of intermediate
  frames. Additionally, we incorporate advanced guidance techniques, CFG++ and
  DDS, to further enhance the interpolation process. By integrating these, our
  method achieves state-of-the-art performance, efficiently generating
  high-quality, smooth videos between keyframes. On a single 3090 GPU, our method
  can interpolate 25 frames at 1024 x 576 resolution in just 195 seconds,
  establishing it as a leading solution for keyframe interpolation.
paper_authors._id: 6707511a95f90bdb8e8dfc5e
paper_authors.name: Serin Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T09:49:23.096Z
paper_authors._id: 63a2b7f9cc214062058e5a81
paper_authors.avatarUrl: /avatars/be49a6e3aa6e4a3ad8800858d9a82ef1.svg
paper_authors.fullname: Serin Yang
paper_authors.type: user
paper_authors.user: sr2851766
paper_authors._id: 6707511a95f90bdb8e8dfc5f
paper_authors.name: Taesung Kwon
paper_authors._id: 6707511a95f90bdb8e8dfc60
paper_authors.name: Jong Chul Ye
paper_title: ViBiDSampler: Enhancing Video Interpolation Using Bidirectional
    Diffusion Sampler
paper_summary: Recent progress in large-scale text-to-video (T2V) and image-to-video (I2V)
  diffusion models has greatly enhanced video generation, especially in terms of
  keyframe interpolation. However, current image-to-video diffusion models, while
  powerful in generating videos from a single conditioning frame, need adaptation
  for two-frame (start & end) conditioned generation, which is essential for
  effective bounded interpolation. Unfortunately, existing approaches that fuse
  temporally forward and backward paths in parallel often suffer from
  off-manifold issues, leading to artifacts or requiring multiple iterative
  re-noising steps. In this work, we introduce a novel, bidirectional sampling
  strategy to address these off-manifold issues without requiring extensive
  re-noising or fine-tuning. Our method employs sequential sampling along both
  forward and backward paths, conditioned on the start and end frames,
  respectively, ensuring more coherent and on-manifold generation of intermediate
  frames. Additionally, we incorporate advanced guidance techniques, CFG++ and
  DDS, to further enhance the interpolation process. By integrating these, our
  method achieves state-of-the-art performance, efficiently generating
  high-quality, smooth videos between keyframes. On a single 3090 GPU, our method
  can interpolate 25 frames at 1024 x 576 resolution in just 195 seconds,
  establishing it as a leading solution for keyframe interpolation.
paper_ai_summary: A bidirectional sampling strategy combined with CFG++ and DDS is introduced to improve image-to-video diffusion models for effective keyframe interpolation, achieving high-quality results quickly.
paper_ai_keywords: text-to-video image-to-video diffusion models keyframe interpolation bidirectional sampling off-manifold issues CFG++ DDS intermediate frames