InstructVideo: Instructing Video Diffusion Models with Human Feedback
Diffusion models have emerged as the de facto paradigm for video generation.
However, their reliance on web-scale data of varied quality often yields
results that are visually unappealing and misaligned with the textual prompts.
To tackle this problem, we propose InstructVideo to instruct text-to-video
diffusion models with human feedback by reward fine-tuning. InstructVideo has
two key ingredients: 1) To ameliorate the cost of reward fine-tuning induced by
generating through the full DDIM sampling chain, we recast reward fine-tuning
as editing. By leveraging the diffusion process to corrupt a sampled video,
InstructVideo requires only partial inference of the DDIM sampling chain,
reducing fine-tuning cost while improving fine-tuning efficiency. 2) To
mitigate the absence of a dedicated video reward model for human preferences,
we repurpose established image reward models, e.g., HPSv2. To this end, we
propose Segmental Video Reward, a mechanism to provide reward signals based on
segmental sparse sampling, and Temporally Attenuated Reward, a method that
mitigates temporal modeling degradation during fine-tuning. Extensive
experiments, both qualitative and quantitative, validate the practicality and
efficacy of using image reward models in InstructVideo, significantly enhancing
the visual quality of generated videos without compromising generalization
capabilities. Code and models will be made publicly available.
6583ad0156d225548b2853c7
Hangjie Yuan
admin_assigned
2023-12-21T08:56:56.982Z
649d54b314afbb10ce2a9eeb
/avatars/15c325d8c2273ff63569f23015e98486.svg
Hangjie Yuan
user
JacobYuan
6583ad0156d225548b2853c8
Shiwei Zhang
admin_assigned
2023-12-21T08:57:18.972Z
63a1200045edac9f7508bae9
/avatars/e84f0b045f32c5b8b4da43458650b925.svg
Shiwei Zhang
user
StevenZhang
6583ad0156d225548b2853c9
Xiang Wang
6583ad0156d225548b2853ca
Yujie Wei
claimed_verified
2024-04-11T07:44:53.686Z
637f70d6fab5db9101c3dfc8
https://cdn-avatars.huggingface.co/v1/production/uploads/637f70d6fab5db9101c3dfc8/NgkYNXWLDavLbrnCby2Fl.jpeg
Yujie Wei
user
weilllllls
6583ad0156d225548b2853cb
Tao Feng
6583ad0156d225548b2853cc
Yining Pan
admin_assigned
2023-12-21T08:58:46.818Z
6337c57d0267ebcf026c0394
/avatars/a706e774614767e3f3f8c9d4be62d350.svg
Yining Pan
user
Closedshop
6583ad0156d225548b2853cd
Yingya Zhang
6583ad0156d225548b2853ce
Ziwei Liu
admin_assigned
2023-12-21T08:59:09.430Z
62ab1ac1d48b4d8b048a3473
https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
Ziwei Liu
user
liuziwei7
6583ad0156d225548b2853cf
Samuel Albanie
admin_assigned
2023-12-21T08:59:19.539Z
62f3efefd6ba2ee26651f44a
https://cdn-avatars.huggingface.co/v1/production/uploads/1660153837083-noauth.png
Samuel Albanie
user
albanie
6583ad0156d225548b2853d0
Dong Ni
InstructVideo: Instructing Video Diffusion Models with Human Feedback
Diffusion models have emerged as the de facto paradigm for video generation.
However, their reliance on web-scale data of varied quality often yields
results that are visually unappealing and misaligned with the textual prompts.
To tackle this problem, we propose InstructVideo to instruct text-to-video
diffusion models with human feedback by reward fine-tuning. InstructVideo has
two key ingredients: 1) To ameliorate the cost of reward fine-tuning induced by
generating through the full DDIM sampling chain, we recast reward fine-tuning
as editing. By leveraging the diffusion process to corrupt a sampled video,
InstructVideo requires only partial inference of the DDIM sampling chain,
reducing fine-tuning cost while improving fine-tuning efficiency. 2) To
mitigate the absence of a dedicated video reward model for human preferences,
we repurpose established image reward models, e.g., HPSv2. To this end, we
propose Segmental Video Reward, a mechanism to provide reward signals based on
segmental sparse sampling, and Temporally Attenuated Reward, a method that
mitigates temporal modeling degradation during fine-tuning. Extensive
experiments, both qualitative and quantitative, validate the practicality and
efficacy of using image reward models in InstructVideo, significantly enhancing
the visual quality of generated videos without compromising generalization
capabilities. Code and models will be made publicly available.
InstructVideo enhances video generation by reward fine-tuning text-to-video diffusion models with human feedback using repurposed image reward models and efficient segmentation techniques.
diffusion models video generation web-scale data textual prompts reward fine-tuning editing DDIM sampling chain partial inference video reward model HPSv2 Segmental Video Reward Temporally Attenuated Reward