title: T2V-Turbo-v2: Enhancing Video Generation Model Post-Training through
    Data, Reward, and Conditional Guidance Design
summary: In this paper, we focus on enhancing a diffusion-based text-to-video (T2V)
  model during the post-training phase by distilling a highly capable consistency
  model from a pretrained T2V model. Our proposed method, T2V-Turbo-v2,
  introduces a significant advancement by integrating various supervision
  signals, including high-quality training data, reward model feedback, and
  conditional guidance, into the consistency distillation process. Through
  comprehensive ablation studies, we highlight the crucial importance of
  tailoring datasets to specific learning objectives and the effectiveness of
  learning from diverse reward models for enhancing both the visual quality and
  text-video alignment. Additionally, we highlight the vast design space of
  conditional guidance strategies, which centers on designing an effective energy
  function to augment the teacher ODE solver. We demonstrate the potential of
  this approach by extracting motion guidance from the training datasets and
  incorporating it into the ODE solver, showcasing its effectiveness in improving
  the motion quality of the generated videos with the improved motion-related
  metrics from VBench and T2V-CompBench. Empirically, our T2V-Turbo-v2
  establishes a new state-of-the-art result on VBench, with a Total score of
  85.13, surpassing proprietary systems such as Gen-3 and Kling.
paper_authors._id: 6707527a2f420b4b07e4da90
paper_authors.name: Jiachen Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-10T08:00:33.880Z
paper_authors._id: 640d7ec5fdeaae13907fc488
paper_authors.avatarUrl: /avatars/acf43ea155105a51c8612dacc4725091.svg
paper_authors.fullname: Jiachen Li
paper_authors.type: user
paper_authors.user: jiachenli-ucsb
paper_authors._id: 6707527a2f420b4b07e4da91
paper_authors.name: Qian Long
paper_authors._id: 6707527a2f420b4b07e4da92
paper_authors.name: Jian Zheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-11T07:44:09.821Z
paper_authors._id: 6424d5b196f39ebc04165381
paper_authors.avatarUrl: /avatars/35c95b1c1449921189585b1606b32db7.svg
paper_authors.fullname: Zheng
paper_authors.type: user
paper_authors.user: JulyApril
paper_authors._id: 6707527a2f420b4b07e4da93
paper_authors.name: Xiaofeng Gao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T09:35:42.663Z
paper_authors._id: 64485435e21484883406f5fd
paper_authors.avatarUrl: /avatars/910b920a6a6e8c7e0c7f7406554dd55b.svg
paper_authors.fullname: Xiaofeng Gao
paper_authors.type: user
paper_authors.user: xfgao
paper_authors._id: 6707527a2f420b4b07e4da94
paper_authors.name: Robinson Piramuthu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T09:35:37.667Z
paper_authors._id: 639a65cdabfa5e2fccf18b82
paper_authors.avatarUrl: /avatars/945f6478d7a219174ffb1ccd14d7bad5.svg
paper_authors.fullname: Robinson Piramuthu
paper_authors.type: user
paper_authors.user: rpiramuthu
paper_authors._id: 6707527a2f420b4b07e4da95
paper_authors.name: Wenhu Chen
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-10-10T04:05:18.584Z
paper_authors._id: 6313a86154e6e5d9f0f94e04
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1662232951344-6313a86154e6e5d9f0f94e04.jpeg
paper_authors.fullname: Wenhu Chen
paper_authors.type: user
paper_authors.user: wenhu
paper_authors._id: 6707527a2f420b4b07e4da96
paper_authors.name: William Yang Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T09:38:43.138Z
paper_authors._id: 6476607efb22e3b77f3f1452
paper_authors.avatarUrl: /avatars/22f40c81e6b6e09bdb66a5afd978f98b.svg
paper_authors.fullname: William Yang Wang
paper_authors.type: user
paper_authors.user: wangwilliamyang
paper_title: T2V-Turbo-v2: Enhancing Video Generation Model Post-Training through
    Data, Reward, and Conditional Guidance Design
paper_summary: In this paper, we focus on enhancing a diffusion-based text-to-video (T2V)
  model during the post-training phase by distilling a highly capable consistency
  model from a pretrained T2V model. Our proposed method, T2V-Turbo-v2,
  introduces a significant advancement by integrating various supervision
  signals, including high-quality training data, reward model feedback, and
  conditional guidance, into the consistency distillation process. Through
  comprehensive ablation studies, we highlight the crucial importance of
  tailoring datasets to specific learning objectives and the effectiveness of
  learning from diverse reward models for enhancing both the visual quality and
  text-video alignment. Additionally, we highlight the vast design space of
  conditional guidance strategies, which centers on designing an effective energy
  function to augment the teacher ODE solver. We demonstrate the potential of
  this approach by extracting motion guidance from the training datasets and
  incorporating it into the ODE solver, showcasing its effectiveness in improving
  the motion quality of the generated videos with the improved motion-related
  metrics from VBench and T2V-CompBench. Empirically, our T2V-Turbo-v2
  establishes a new state-of-the-art result on VBench, with a Total score of
  85.13, surpassing proprietary systems such as Gen-3 and Kling.
paper_ai_keywords: diffusion-based text-to-video (T2V) model consistency model pretrained T2V model T2V-Turbo-v2 supervision signals high-quality training data reward model feedback conditional guidance consistency distillation effective energy function teacher ODE solver motion guidance motion quality VBench T2V-CompBench Total score proprietary systems Gen-3 Kling