Sci-Fi: Symmetric Constraint for Frame Inbetweening
Frame inbetweening aims to synthesize intermediate video sequences
conditioned on the given start and end frames. Current state-of-the-art methods
mainly extend large-scale pre-trained Image-to-Video Diffusion models (I2V-DMs)
by incorporating end-frame constraints via directly fine-tuning or omitting
training. We identify a critical limitation in their design: Their injections
of the end-frame constraint usually utilize the same mechanism that originally
imposed the start-frame (single image) constraint. However, since the original
I2V-DMs are adequately trained for the start-frame condition in advance,
naively introducing the end-frame constraint by the same mechanism with much
less (even zero) specialized training probably can't make the end frame have a
strong enough impact on the intermediate content like the start frame. This
asymmetric control strength of the two frames over the intermediate content
likely leads to inconsistent motion or appearance collapse in generated frames.
To efficiently achieve symmetric constraints of start and end frames, we
propose a novel framework, termed Sci-Fi, which applies a stronger injection
for the constraint of a smaller training scale. Specifically, it deals with the
start-frame constraint as before, while introducing the end-frame constraint by
an improved mechanism. The new mechanism is based on a well-designed
lightweight module, named EF-Net, which encodes only the end frame and expands
it into temporally adaptive frame-wise features injected into the I2V-DM. This
makes the end-frame constraint as strong as the start-frame constraint,
enabling our Sci-Fi to produce more harmonious transitions in various
scenarios. Extensive experiments prove the superiority of our Sci-Fi compared
with other baselines.
68367078bec6153cee9be84c
Liuhan Chen
68367078bec6153cee9be84d
Xiaodong Cun
claimed_verified
2025-06-30T06:23:37.244Z
63184c517ca1b876d99b7e0e
https://cdn-avatars.huggingface.co/v1/production/uploads/63184c517ca1b876d99b7e0e/b-qDExoeJuDXK0cJBZKnz.jpeg
Xiaodong Cun
user
vinthony
68367078bec6153cee9be84e
Xiaoyu Li
68367078bec6153cee9be84f
Xianyi He
68367078bec6153cee9be850
Shenghai Yuan
claimed_verified
2025-05-28T08:58:01.230Z
63468720dd6d90d82ccf3450
https://cdn-avatars.huggingface.co/v1/production/uploads/63468720dd6d90d82ccf3450/tVBFlmZNz8FRMkOrDaDID.jpeg
YSH
user
BestWishYsh
68367078bec6153cee9be851
Jie Chen
68367078bec6153cee9be852
Ying Shan
68367078bec6153cee9be853
Li Yuan
Sci-Fi: Symmetric Constraint for Frame Inbetweening
Frame inbetweening aims to synthesize intermediate video sequences
conditioned on the given start and end frames. Current state-of-the-art methods
mainly extend large-scale pre-trained Image-to-Video Diffusion models (I2V-DMs)
by incorporating end-frame constraints via directly fine-tuning or omitting
training. We identify a critical limitation in their design: Their injections
of the end-frame constraint usually utilize the same mechanism that originally
imposed the start-frame (single image) constraint. However, since the original
I2V-DMs are adequately trained for the start-frame condition in advance,
naively introducing the end-frame constraint by the same mechanism with much
less (even zero) specialized training probably can't make the end frame have a
strong enough impact on the intermediate content like the start frame. This
asymmetric control strength of the two frames over the intermediate content
likely leads to inconsistent motion or appearance collapse in generated frames.
To efficiently achieve symmetric constraints of start and end frames, we
propose a novel framework, termed Sci-Fi, which applies a stronger injection
for the constraint of a smaller training scale. Specifically, it deals with the
start-frame constraint as before, while introducing the end-frame constraint by
an improved mechanism. The new mechanism is based on a well-designed
lightweight module, named EF-Net, which encodes only the end frame and expands
it into temporally adaptive frame-wise features injected into the I2V-DM. This
makes the end-frame constraint as strong as the start-frame constraint,
enabling our Sci-Fi to produce more harmonious transitions in various
scenarios. Extensive experiments prove the superiority of our Sci-Fi compared
with other baselines.
A novel framework named Sci-Fi addresses the inconsistency in frame control strength by introducing a stronger end-frame constraint mechanism, improving harmonious transitions in frame inbetweening.
frame inbetweening Image-to-Video Diffusion models I2V-DMs end-frame constraints start-frame constraint asymmetric control strength consistent motion appearance collapse EF-Net temporally adaptive frame-wise features