VideoGuide: Improving Video Diffusion Models without Training Through a
  Teacher's Guide
Text-to-image (T2I) diffusion models have revolutionized visual content
creation, but extending these capabilities to text-to-video (T2V) generation
remains a challenge, particularly in preserving temporal consistency. Existing
methods that aim to improve consistency often cause trade-offs such as reduced
imaging quality and impractical computational time. To address these issues we
introduce VideoGuide, a novel framework that enhances the temporal consistency
of pretrained T2V models without the need for additional training or
fine-tuning. Instead, VideoGuide leverages any pretrained video diffusion model
(VDM) or itself as a guide during the early stages of inference, improving
temporal quality by interpolating the guiding model's denoised samples into the
sampling model's denoising process. The proposed method brings about
significant improvement in temporal consistency and image fidelity, providing a
cost-effective and practical solution that synergizes the strengths of various
video diffusion models. Furthermore, we demonstrate prior distillation,
revealing that base models can achieve enhanced text coherence by utilizing the
superior data prior of the guiding model through the proposed method. Project
Page: http://videoguide2025.github.io/
6704b04606af66752cd6a20a
Dohun Lee
claimed_verified
2024-10-08T09:05:33.908Z
665b04fdef08dd8cf4b7579e
/avatars/02694064a1508503a6b58f2c5b9e74ae.svg
Dohun Lee
user
DHCAI
6704b04606af66752cd6a20b
Bryan S Kim
claimed_verified
2024-10-08T08:17:25.231Z
6628efe14e1fa854f48d3a28
/avatars/aa5421149a07a82b5c2a25978f9b6926.svg
Sangwoo Kim
user
bryanswkim
6704b04606af66752cd6a20c
Geon Yeong Park
claimed_verified
2024-10-08T08:17:26.806Z
64cfff0abf39f9c8bebd2a6f
/avatars/e3405136c8af5065c287eff718c78b5b.svg
Geon Yeong Park
user
geonyoung-park
6704b04606af66752cd6a20d
Jong Chul Ye
VideoGuide: Improving Video Diffusion Models without Training Through a
  Teacher's Guide
Text-to-image (T2I) diffusion models have revolutionized visual content
creation, but extending these capabilities to text-to-video (T2V) generation
remains a challenge, particularly in preserving temporal consistency. Existing
methods that aim to improve consistency often cause trade-offs such as reduced
imaging quality and impractical computational time. To address these issues we
introduce VideoGuide, a novel framework that enhances the temporal consistency
of pretrained T2V models without the need for additional training or
fine-tuning. Instead, VideoGuide leverages any pretrained video diffusion model
(VDM) or itself as a guide during the early stages of inference, improving
temporal quality by interpolating the guiding model's denoised samples into the
sampling model's denoising process. The proposed method brings about
significant improvement in temporal consistency and image fidelity, providing a
cost-effective and practical solution that synergizes the strengths of various
video diffusion models. Furthermore, we demonstrate prior distillation,
revealing that base models can achieve enhanced text coherence by utilizing the
superior data prior of the guiding model through the proposed method. Project
Page: http://videoguide2025.github.io/
VideoGuide enhances temporal consistency and image fidelity in text-to-video generation by leveraging pretrained video diffusion models during the inference process without additional training.
text-to-image models text-to-video models temporal consistency pretrained video diffusion models denoised samples sampling model prior distillation text coherence