title: Training-free Long Video Generation with Chain of Diffusion Model
    Experts
summary: Video generation models hold substantial potential in areas such as
  filmmaking. However, current video diffusion models need high computational
  costs and produce suboptimal results due to high complexity of video generation
  task. In this paper, we propose ConFiner, an efficient high-quality
  video generation framework that decouples video generation into easier
  subtasks: structure control and spatial-temporal refinement.
  It can generate high-quality videos with chain of off-the-shelf diffusion model
  experts, each expert responsible for a decoupled subtask. During the
  refinement, we introduce coordinated denoising, which can merge multiple
  diffusion experts' capabilities into a single sampling. Furthermore, we design
  ConFiner-Long framework, which can generate long coherent video with three
  constraint strategies on ConFiner. Experimental results indicate that with only
  10\% of the inference cost, our ConFiner surpasses representative models like
  Lavie and Modelscope across all objective and subjective metrics. And
  ConFiner-Long can generate high-quality and coherent videos with up to 600
  frames.
paper_authors._id: 66cd3b95a9107693c2df2f09
paper_authors.name: Wenhao Li
paper_authors._id: 66cd3b95a9107693c2df2f0a
paper_authors.name: Yichao Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-27T12:39:53.673Z
paper_authors._id: 65a8bf99c5bf09bc93770320
paper_authors.avatarUrl: /avatars/baf00b5902d0a75291d22477b23a27c4.svg
paper_authors.fullname: caoyichao
paper_authors.type: user
paper_authors.user: caoyichao
paper_authors._id: 66cd3b95a9107693c2df2f0b
paper_authors.name: Xie Su
paper_authors._id: 66cd3b95a9107693c2df2f0c
paper_authors.name: Xi Lin
paper_authors._id: 66cd3b95a9107693c2df2f0d
paper_authors.name: Shan You
paper_authors._id: 66cd3b95a9107693c2df2f0e
paper_authors.name: Mingkai Zheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-27T12:40:26.570Z
paper_authors._id: 65119c5560584e99b153eb25
paper_authors.avatarUrl: /avatars/7a6156b31402851831a4d3edbff49abf.svg
paper_authors.fullname: Mingkai Zheng
paper_authors.type: user
paper_authors.user: Kylez97
paper_authors._id: 66cd3b95a9107693c2df2f0f
paper_authors.name: Yi Chen
paper_authors._id: 66cd3b95a9107693c2df2f10
paper_authors.name: Chang Xu
paper_title: Training-free Long Video Generation with Chain of Diffusion Model
    Experts
paper_summary: Video generation models hold substantial potential in areas such as
  filmmaking. However, current video diffusion models need high computational
  costs and produce suboptimal results due to high complexity of video generation
  task. In this paper, we propose ConFiner, an efficient high-quality
  video generation framework that decouples video generation into easier
  subtasks: structure control and spatial-temporal refinement.
  It can generate high-quality videos with chain of off-the-shelf diffusion model
  experts, each expert responsible for a decoupled subtask. During the
  refinement, we introduce coordinated denoising, which can merge multiple
  diffusion experts' capabilities into a single sampling. Furthermore, we design
  ConFiner-Long framework, which can generate long coherent video with three
  constraint strategies on ConFiner. Experimental results indicate that with only
  10\% of the inference cost, our ConFiner surpasses representative models like
  Lavie and Modelscope across all objective and subjective metrics. And
  ConFiner-Long can generate high-quality and coherent videos with up to 600
  frames.
paper_ai_keywords: diffusion models video diffusion models ConFiner visual control spatial-temporal refinement coordinated denoising ConFiner-Long constraint strategies inference cost Lavie Modelscope