Explorative Inbetweening of Time and Space
We introduce bounded generation as a generalized task to control video
generation to synthesize arbitrary camera and subject motion based only on a
given start and end frame. Our objective is to fully leverage the inherent
generalization capability of an image-to-video model without additional
training or fine-tuning of the original model. This is achieved through the
proposed new sampling strategy, which we call Time Reversal Fusion, that fuses
the temporally forward and backward denoising paths conditioned on the start
and end frame, respectively. The fused path results in a video that smoothly
connects the two frames, generating inbetweening of faithful subject motion,
novel views of static scenes, and seamless video looping when the two bounding
frames are identical. We curate a diverse evaluation dataset of image pairs and
compare against the closest existing methods. We find that Time Reversal Fusion
outperforms related work on all subtasks, exhibiting the ability to generate
complex motions and 3D-consistent views guided by bounded frames. See project
page at https://time-reversal.github.io.
65fced62afa13a681664b091
Haiwen Feng
claimed_verified
2024-03-22T13:24:57.590Z
644d2ede5c1b4e14d0c065af
/avatars/34945e0ab925d8eabc6580c71e053b11.svg
Haiwen (Haven) Feng
user
havenfeng
65fced62afa13a681664b092
Zheng Ding
65fced62afa13a681664b093
Zhihao Xia
admin_assigned
2024-03-22T10:26:34.745Z
64df0d5a1961da512204db2c
/avatars/e08625382ba8244b72d2e94a11ff325a.svg
xiazhihao
user
xiazhihao
65fced62afa13a681664b094
Simon Niklaus
65fced62afa13a681664b095
Victoria Abrevaya
65fced62afa13a681664b096
Michael J. Black
admin_assigned
2024-03-22T10:27:04.331Z
649fbababa7a05e3626546c8
/avatars/a9aa4b4914649527094fc1b12779e014.svg
Michael J. Black
user
MichaelJBlack
65fced62afa13a681664b097
Xuaner Zhang
Explorative Inbetweening of Time and Space
We introduce bounded generation as a generalized task to control video
generation to synthesize arbitrary camera and subject motion based only on a
given start and end frame. Our objective is to fully leverage the inherent
generalization capability of an image-to-video model without additional
training or fine-tuning of the original model. This is achieved through the
proposed new sampling strategy, which we call Time Reversal Fusion, that fuses
the temporally forward and backward denoising paths conditioned on the start
and end frame, respectively. The fused path results in a video that smoothly
connects the two frames, generating inbetweening of faithful subject motion,
novel views of static scenes, and seamless video looping when the two bounding
frames are identical. We curate a diverse evaluation dataset of image pairs and
compare against the closest existing methods. We find that Time Reversal Fusion
outperforms related work on all subtasks, exhibiting the ability to generate
complex motions and 3D-consistent views guided by bounded frames. See project
page at https://time-reversal.github.io.
Time Reversal Fusion enables video generation by fusing forward and backward denoising paths conditioned on start and end frames, outperforming existing methods in generating smooth, complex motions, and 3D-consistent views.
bounded generation image-to-video model Time Reversal Fusion denoising paths inbetweening seamless video looping evaluation dataset