title: InstructVideo: Instructing Video Diffusion Models with Human Feedback
summary: Diffusion models have emerged as the de facto paradigm for video generation.
  However, their reliance on web-scale data of varied quality often yields
  results that are visually unappealing and misaligned with the textual prompts.
  To tackle this problem, we propose InstructVideo to instruct text-to-video
  diffusion models with human feedback by reward fine-tuning. InstructVideo has
  two key ingredients: 1) To ameliorate the cost of reward fine-tuning induced by
  generating through the full DDIM sampling chain, we recast reward fine-tuning
  as editing. By leveraging the diffusion process to corrupt a sampled video,
  InstructVideo requires only partial inference of the DDIM sampling chain,
  reducing fine-tuning cost while improving fine-tuning efficiency. 2) To
  mitigate the absence of a dedicated video reward model for human preferences,
  we repurpose established image reward models, e.g., HPSv2. To this end, we
  propose Segmental Video Reward, a mechanism to provide reward signals based on
  segmental sparse sampling, and Temporally Attenuated Reward, a method that
  mitigates temporal modeling degradation during fine-tuning. Extensive
  experiments, both qualitative and quantitative, validate the practicality and
  efficacy of using image reward models in InstructVideo, significantly enhancing
  the visual quality of generated videos without compromising generalization
  capabilities. Code and models will be made publicly available.
paper_authors._id: 6583ad0156d225548b2853c7
paper_authors.name: Hangjie Yuan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-21T08:56:56.982Z
paper_authors._id: 649d54b314afbb10ce2a9eeb
paper_authors.avatarUrl: /avatars/15c325d8c2273ff63569f23015e98486.svg
paper_authors.fullname: Hangjie Yuan
paper_authors.type: user
paper_authors.user: JacobYuan
paper_authors._id: 6583ad0156d225548b2853c8
paper_authors.name: Shiwei Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-21T08:57:18.972Z
paper_authors._id: 63a1200045edac9f7508bae9
paper_authors.avatarUrl: /avatars/e84f0b045f32c5b8b4da43458650b925.svg
paper_authors.fullname: Shiwei Zhang
paper_authors.type: user
paper_authors.user: StevenZhang
paper_authors._id: 6583ad0156d225548b2853c9
paper_authors.name: Xiang Wang
paper_authors._id: 6583ad0156d225548b2853ca
paper_authors.name: Yujie Wei
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-11T07:44:53.686Z
paper_authors._id: 637f70d6fab5db9101c3dfc8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/637f70d6fab5db9101c3dfc8/NgkYNXWLDavLbrnCby2Fl.jpeg
paper_authors.fullname: Yujie Wei
paper_authors.type: user
paper_authors.user: weilllllls
paper_authors._id: 6583ad0156d225548b2853cb
paper_authors.name: Tao Feng
paper_authors._id: 6583ad0156d225548b2853cc
paper_authors.name: Yining Pan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-21T08:58:46.818Z
paper_authors._id: 6337c57d0267ebcf026c0394
paper_authors.avatarUrl: /avatars/a706e774614767e3f3f8c9d4be62d350.svg
paper_authors.fullname: Yining Pan
paper_authors.type: user
paper_authors.user: Closedshop
paper_authors._id: 6583ad0156d225548b2853cd
paper_authors.name: Yingya Zhang
paper_authors._id: 6583ad0156d225548b2853ce
paper_authors.name: Ziwei Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-21T08:59:09.430Z
paper_authors._id: 62ab1ac1d48b4d8b048a3473
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
paper_authors.fullname: Ziwei Liu
paper_authors.type: user
paper_authors.user: liuziwei7
paper_authors._id: 6583ad0156d225548b2853cf
paper_authors.name: Samuel Albanie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-21T08:59:19.539Z
paper_authors._id: 62f3efefd6ba2ee26651f44a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1660153837083-noauth.png
paper_authors.fullname: Samuel Albanie
paper_authors.type: user
paper_authors.user: albanie
paper_authors._id: 6583ad0156d225548b2853d0
paper_authors.name: Dong Ni
paper_title: InstructVideo: Instructing Video Diffusion Models with Human Feedback
paper_summary: Diffusion models have emerged as the de facto paradigm for video generation.
  However, their reliance on web-scale data of varied quality often yields
  results that are visually unappealing and misaligned with the textual prompts.
  To tackle this problem, we propose InstructVideo to instruct text-to-video
  diffusion models with human feedback by reward fine-tuning. InstructVideo has
  two key ingredients: 1) To ameliorate the cost of reward fine-tuning induced by
  generating through the full DDIM sampling chain, we recast reward fine-tuning
  as editing. By leveraging the diffusion process to corrupt a sampled video,
  InstructVideo requires only partial inference of the DDIM sampling chain,
  reducing fine-tuning cost while improving fine-tuning efficiency. 2) To
  mitigate the absence of a dedicated video reward model for human preferences,
  we repurpose established image reward models, e.g., HPSv2. To this end, we
  propose Segmental Video Reward, a mechanism to provide reward signals based on
  segmental sparse sampling, and Temporally Attenuated Reward, a method that
  mitigates temporal modeling degradation during fine-tuning. Extensive
  experiments, both qualitative and quantitative, validate the practicality and
  efficacy of using image reward models in InstructVideo, significantly enhancing
  the visual quality of generated videos without compromising generalization
  capabilities. Code and models will be made publicly available.
paper_ai_keywords: diffusion models video generation web-scale data textual prompts reward fine-tuning DDIM sampling chain editing video reward model human preferences image reward models Segmental Video Remed segmental sampling temporal modeling degradation generalization capabilities