title: DCM: Dual-Expert Consistency Model for Efficient and High-Quality Video
    Generation
summary: Diffusion Models have achieved remarkable results in video synthesis but
  require iterative denoising steps, leading to substantial computational
  overhead. Consistency Models have made significant progress in accelerating
  diffusion models. However, directly applying them to video diffusion models
  often results in severe degradation of temporal consistency and appearance
  details. In this paper, by analyzing the training dynamics of Consistency
  Models, we identify a key conflicting learning dynamics during the distillation
  process: there is a significant discrepancy in the optimization gradients and
  loss contributions across different timesteps. This discrepancy prevents the
  distilled student model from achieving an optimal state, leading to compromised
  temporal consistency and degraded appearance details. To address this issue, we
  propose a parameter-efficient Dual-Expert Consistency Model~(DCM),
  where a semantic expert focuses on learning semantic layout and motion, while a
  detail expert specializes in fine detail refinement. Furthermore, we introduce
  Temporal Coherence Loss to improve motion consistency for the semantic expert
  and apply GAN and Feature Matching Loss to enhance the synthesis quality of the
  detail expert.Our approach achieves state-of-the-art visual quality with
  significantly reduced sampling steps, demonstrating the effectiveness of expert
  specialization in video diffusion model distillation. Our code and models are
  available at
  https://github.com/Vchitect/DCM{https://github.com/Vchitect/DCM}.
paper_authors._id: 684050b04e6c9ec4722c2154
paper_authors.name: Zhengyao Lv
paper_authors._id: 684050b04e6c9ec4722c2155
paper_authors.name: Chenyang Si
paper_authors._id: 684050b04e6c9ec4722c2156
paper_authors.name: Tianlin Pan
paper_authors._id: 684050b04e6c9ec4722c2157
paper_authors.name: Zhaoxi Chen
paper_authors._id: 684050b04e6c9ec4722c2158
paper_authors.name: Kwan-Yee K. Wong
paper_authors._id: 684050b04e6c9ec4722c2159
paper_authors.name: Yu Qiao
paper_authors._id: 684050b04e6c9ec4722c215a
paper_authors.name: Ziwei Liu
paper_title: DCM: Dual-Expert Consistency Model for Efficient and High-Quality Video
    Generation
paper_summary: Diffusion Models have achieved remarkable results in video synthesis but
  require iterative denoising steps, leading to substantial computational
  overhead. Consistency Models have made significant progress in accelerating
  diffusion models. However, directly applying them to video diffusion models
  often results in severe degradation of temporal consistency and appearance
  details. In this paper, by analyzing the training dynamics of Consistency
  Models, we identify a key conflicting learning dynamics during the distillation
  process: there is a significant discrepancy in the optimization gradients and
  loss contributions across different timesteps. This discrepancy prevents the
  distilled student model from achieving an optimal state, leading to compromised
  temporal consistency and degraded appearance details. To address this issue, we
  propose a parameter-efficient Dual-Expert Consistency Model~(DCM),
  where a semantic expert focuses on learning semantic layout and motion, while a
  detail expert specializes in fine detail refinement. Furthermore, we introduce
  Temporal Coherence Loss to improve motion consistency for the semantic expert
  and apply GAN and Feature Matching Loss to enhance the synthesis quality of the
  detail expert.Our approach achieves state-of-the-art visual quality with
  significantly reduced sampling steps, demonstrating the effectiveness of expert
  specialization in video diffusion model distillation. Our code and models are
  available at
  https://github.com/Vchitect/DCM{https://github.com/Vchitect/DCM}.
paper_ai_summary: The Dual-Expert Consistency Model improves video diffusion model distillation by addressing learning dynamics issues, achieving state-of-the-art quality with reduced sampling steps.
paper_ai_keywords: Diffusion Models Consistency Models training dynamics dual-expert semantic expert detail expert Temporal Coherence Loss GAN Feature Matching Loss