title: LiFT: Leveraging Human Feedback for Text-to-Video Model Alignment
summary: Recent advancements in text-to-video (T2V) generative models have shown
  impressive capabilities. However, these models are still inadequate in aligning
  synthesized videos with human preferences (e.g., accurately reflecting text
  descriptions), which is particularly difficult to address, as human preferences
  are inherently subjective and challenging to formalize as objective functions.
  Therefore, this paper proposes LiFT, a novel fine-tuning method leveraging
  human feedback for T2V model alignment. Specifically, we first construct a
  Human Rating Annotation dataset, LiFT-HRA, consisting of approximately 10k
  human annotations, each including a score and its corresponding rationale.
  Based on this, we train a reward model LiFT-Critic to learn reward function
  effectively, which serves as a proxy for human judgment, measuring the
  alignment between given videos and human expectations. Lastly, we leverage the
  learned reward function to align the T2V model by maximizing the
  reward-weighted likelihood. As a case study, we apply our pipeline to
  CogVideoX-2B, showing that the fine-tuned model outperforms the CogVideoX-5B
  across all 16 metrics, highlighting the potential of human feedback in
  improving the alignment and quality of synthesized videos.
paper_authors._id: 67564cddee0958ae76fa740d
paper_authors.name: Yibin Wang
paper_authors._id: 67564cddee0958ae76fa740e
paper_authors.name: Zhiyu Tan
paper_authors._id: 67564cddee0958ae76fa740f
paper_authors.name: Junyan Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-09T14:57:35.079Z
paper_authors._id: 64dc6d60d8d7d6d6d9f3c78b
paper_authors.avatarUrl: /avatars/b48a8760cd56d4b7b579f235be5637f3.svg
paper_authors.fullname: Junyan Wang
paper_authors.type: user
paper_authors.user: HotWater666
paper_authors._id: 67564cddee0958ae76fa7410
paper_authors.name: Xiaomeng Yang
paper_authors._id: 67564cddee0958ae76fa7411
paper_authors.name: Cheng Jin
paper_authors._id: 67564cddee0958ae76fa7412
paper_authors.name: Hao Li
paper_title: LiFT: Leveraging Human Feedback for Text-to-Video Model Alignment
paper_summary: Recent advancements in text-to-video (T2V) generative models have shown
  impressive capabilities. However, these models are still inadequate in aligning
  synthesized videos with human preferences (e.g., accurately reflecting text
  descriptions), which is particularly difficult to address, as human preferences
  are inherently subjective and challenging to formalize as objective functions.
  Therefore, this paper proposes LiFT, a novel fine-tuning method leveraging
  human feedback for T2V model alignment. Specifically, we first construct a
  Human Rating Annotation dataset, LiFT-HRA, consisting of approximately 10k
  human annotations, each including a score and its corresponding rationale.
  Based on this, we train a reward model LiFT-Critic to learn reward function
  effectively, which serves as a proxy for human judgment, measuring the
  alignment between given videos and human expectations. Lastly, we leverage the
  learned reward function to align the T2V model by maximizing the
  reward-weighted likelihood. As a case study, we apply our pipeline to
  CogVideoX-2B, showing that the fine-tuned model outperforms the CogVideoX-5B
  across all 16 metrics, highlighting the potential of human feedback in
  improving the alignment and quality of synthesized videos.
paper_ai_keywords: text-to-video (T2V) generative models human preferences fine-tuning method Human Rating Annotation dataset (LiFT-HRA) HumanAnnotations reward model (LiFT-Critic) reward function human judgment reward-weighted likelihood CogVideoX-2B CogVideoX-5B synthesized videos human expectations alignment and quality