title: Explorative Inbetweening of Time and Space
summary: We introduce bounded generation as a generalized task to control video
  generation to synthesize arbitrary camera and subject motion based only on a
  given start and end frame. Our objective is to fully leverage the inherent
  generalization capability of an image-to-video model without additional
  training or fine-tuning of the original model. This is achieved through the
  proposed new sampling strategy, which we call Time Reversal Fusion, that fuses
  the temporally forward and backward denoising paths conditioned on the start
  and end frame, respectively. The fused path results in a video that smoothly
  connects the two frames, generating inbetweening of faithful subject motion,
  novel views of static scenes, and seamless video looping when the two bounding
  frames are identical. We curate a diverse evaluation dataset of image pairs and
  compare against the closest existing methods. We find that Time Reversal Fusion
  outperforms related work on all subtasks, exhibiting the ability to generate
  complex motions and 3D-consistent views guided by bounded frames. See project
  page at https://time-reversal.github.io.
paper_authors._id: 65fced62afa13a681664b091
paper_authors.name: Haiwen Feng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-22T13:24:57.590Z
paper_authors._id: 644d2ede5c1b4e14d0c065af
paper_authors.avatarUrl: /avatars/34945e0ab925d8eabc6580c71e053b11.svg
paper_authors.fullname: Haiwen (Haven) Feng
paper_authors.type: user
paper_authors.user: havenfeng
paper_authors._id: 65fced62afa13a681664b092
paper_authors.name: Zheng Ding
paper_authors._id: 65fced62afa13a681664b093
paper_authors.name: Zhihao Xia
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-22T10:26:34.745Z
paper_authors._id: 64df0d5a1961da512204db2c
paper_authors.avatarUrl: /avatars/e08625382ba8244b72d2e94a11ff325a.svg
paper_authors.fullname: xiazhihao
paper_authors.type: user
paper_authors.user: xiazhihao
paper_authors._id: 65fced62afa13a681664b094
paper_authors.name: Simon Niklaus
paper_authors._id: 65fced62afa13a681664b095
paper_authors.name: Victoria Abrevaya
paper_authors._id: 65fced62afa13a681664b096
paper_authors.name: Michael J. Black
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-22T10:27:04.331Z
paper_authors._id: 649fbababa7a05e3626546c8
paper_authors.avatarUrl: /avatars/a9aa4b4914649527094fc1b12779e014.svg
paper_authors.fullname: Michael J. Black
paper_authors.type: user
paper_authors.user: MichaelJBlack
paper_authors._id: 65fced62afa13a681664b097
paper_authors.name: Xuaner Zhang
paper_title: Explorative Inbetweening of Time and Space
paper_summary: We introduce bounded generation as a generalized task to control video
  generation to synthesize arbitrary camera and subject motion based only on a
  given start and end frame. Our objective is to fully leverage the inherent
  generalization capability of an image-to-video model without additional
  training or fine-tuning of the original model. This is achieved through the
  proposed new sampling strategy, which we call Time Reversal Fusion, that fuses
  the temporally forward and backward denoising paths conditioned on the start
  and end frame, respectively. The fused path results in a video that smoothly
  connects the two frames, generating inbetweening of faithful subject motion,
  novel views of static scenes, and seamless video looping when the two bounding
  frames are identical. We curate a diverse evaluation dataset of image pairs and
  compare against the closest existing methods. We find that Time Reversal Fusion
  outperforms related work on all subtasks, exhibiting the ability to generate
  complex motions and 3D-consistent views guided by bounded frames. See project
  page at https://time-reversal.github.io.
paper_ai_summary: Time Reversal Fusion enables video generation by fusing forward and backward denoising paths conditioned on start and end frames, outperforming existing methods in generating smooth, complex motions, and 3D-consistent views.
paper_ai_keywords: bounded generation image-to-video model Time Reversal Fusion denoising paths inbetweening seamless video looping evaluation dataset