title: I4VGen: Image as Stepping Stone for Text-to-Video Generation
summary: Text-to-video generation has lagged behind text-to-image synthesis in quality
  and diversity due to the complexity of spatio-temporal modeling and limited
  video-text datasets. This paper presents I4VGen, a training-free and
  plug-and-play video diffusion inference framework, which enhances text-to-video
  generation by leveraging robust image techniques. Specifically, following
  text-to-image-to-video, I4VGen decomposes the text-to-video generation into two
  stages: anchor image synthesis and anchor image-guided video synthesis.
  Correspondingly, a well-designed generation-selection pipeline is employed to
  achieve visually-realistic and semantically-faithful anchor image, and an
  innovative Noise-Invariant Video Score Distillation Sampling is incorporated to
  animate the image to a dynamic video, followed by a video regeneration process
  to refine the video. This inference strategy effectively mitigates the
  prevalent issue of non-zero terminal signal-to-noise ratio. Extensive
  evaluations show that I4VGen not only produces videos with higher visual
  realism and textual fidelity but also integrates seamlessly into existing
  image-to-video diffusion models, thereby improving overall video quality.
paper_authors._id: 665fe3bc22834190f7cd102a
paper_authors.name: Xiefan Guo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-05T08:24:37.115Z
paper_authors._id: 64905cd589f22918ecaca080
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/2S7I7uZL49CXbUN2T7p63.jpeg
paper_authors.fullname: Xiefan Guo
paper_authors.type: user
paper_authors.user: xiefan-guo
paper_authors._id: 665fe3bc22834190f7cd102b
paper_authors.name: Jinlin Liu
paper_authors._id: 665fe3bc22834190f7cd102c
paper_authors.name: Miaomiao Cui
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-05T08:25:44.412Z
paper_authors._id: 665024770a7476925c31aab8
paper_authors.avatarUrl: /avatars/ab738817124b1a5b2bd7cf9d5bacf1e9.svg
paper_authors.fullname: Cui_Miaomiao
paper_authors.type: user
paper_authors.user: Miaomiao0109
paper_authors._id: 665fe3bc22834190f7cd102d
paper_authors.name: Di Huang
paper_title: I4VGen: Image as Stepping Stone for Text-to-Video Generation
paper_summary: Text-to-video generation has lagged behind text-to-image synthesis in quality
  and diversity due to the complexity of spatio-temporal modeling and limited
  video-text datasets. This paper presents I4VGen, a training-free and
  plug-and-play video diffusion inference framework, which enhances text-to-video
  generation by leveraging robust image techniques. Specifically, following
  text-to-image-to-video, I4VGen decomposes the text-to-video generation into two
  stages: anchor image synthesis and anchor image-guided video synthesis.
  Correspondingly, a well-designed generation-selection pipeline is employed to
  achieve visually-realistic and semantically-faithful anchor image, and an
  innovative Noise-Invariant Video Score Distillation Sampling is incorporated to
  animate the image to a dynamic video, followed by a video regeneration process
  to refine the video. This inference strategy effectively mitigates the
  prevalent issue of non-zero terminal signal-to-noise ratio. Extensive
  evaluations show that I4VGen not only produces videos with higher visual
  realism and textual fidelity but also integrates seamlessly into existing
  image-to-video diffusion models, thereby improving overall video quality.
paper_ai_keywords: text-to-image-to-video anchor image synthesis anchor image-guided video synthesis generation-selection pipeline Noise-Invariant Video Score Distillation Sampling video diffusion inference framework text-to-video generation video regeneration process non-zero terminal signal-to-noise ratio visual realism textual fidelity