Training-free Long Video Generation with Chain of Diffusion Model
  Experts
Video generation models hold substantial potential in areas such as
filmmaking. However, current video diffusion models need high computational
costs and produce suboptimal results due to high complexity of video generation
task. In this paper, we propose ConFiner, an efficient high-quality
video generation framework that decouples video generation into easier
subtasks: structure control and spatial-temporal refinement.
It can generate high-quality videos with chain of off-the-shelf diffusion model
experts, each expert responsible for a decoupled subtask. During the
refinement, we introduce coordinated denoising, which can merge multiple
diffusion experts' capabilities into a single sampling. Furthermore, we design
ConFiner-Long framework, which can generate long coherent video with three
constraint strategies on ConFiner. Experimental results indicate that with only
10\% of the inference cost, our ConFiner surpasses representative models like
Lavie and Modelscope across all objective and subjective metrics. And
ConFiner-Long can generate high-quality and coherent videos with up to 600
frames.
66cd3b95a9107693c2df2f09
Wenhao Li
66cd3b95a9107693c2df2f0a
Yichao Cao
admin_assigned
2024-08-27T12:39:53.673Z
65a8bf99c5bf09bc93770320
/avatars/baf00b5902d0a75291d22477b23a27c4.svg
caoyichao
user
caoyichao
66cd3b95a9107693c2df2f0b
Xie Su
66cd3b95a9107693c2df2f0c
Xi Lin
66cd3b95a9107693c2df2f0d
Shan You
66cd3b95a9107693c2df2f0e
Mingkai Zheng
admin_assigned
2024-08-27T12:40:26.570Z
65119c5560584e99b153eb25
/avatars/7a6156b31402851831a4d3edbff49abf.svg
Mingkai Zheng
user
Kylez97
66cd3b95a9107693c2df2f0f
Yi Chen
66cd3b95a9107693c2df2f10
Chang Xu
Training-free Long Video Generation with Chain of Diffusion Model
  Experts
Video generation models hold substantial potential in areas such as
filmmaking. However, current video diffusion models need high computational
costs and produce suboptimal results due to high complexity of video generation
task. In this paper, we propose ConFiner, an efficient high-quality
video generation framework that decouples video generation into easier
subtasks: structure control and spatial-temporal refinement.
It can generate high-quality videos with chain of off-the-shelf diffusion model
experts, each expert responsible for a decoupled subtask. During the
refinement, we introduce coordinated denoising, which can merge multiple
diffusion experts' capabilities into a single sampling. Furthermore, we design
ConFiner-Long framework, which can generate long coherent video with three
constraint strategies on ConFiner. Experimental results indicate that with only
10\% of the inference cost, our ConFiner surpasses representative models like
Lavie and Modelscope across all objective and subjective metrics. And
ConFiner-Long can generate high-quality and coherent videos with up to 600
frames.
ConFiner is an efficient video generation framework that decouples video creation into control and refinement using diffusion models, reducing computational costs while maintaining high quality and coherence.
video diffusion models ConFiner structure control spatial-temporal refinement diffusion model experts coordinated denoising ConFiner-Long constraint strategies