StableV2V: Stablizing Shape Consistency in Video-to-Video Editing
Recent advancements of generative AI have significantly promoted content
creation and editing, where prevailing studies further extend this exciting
progress to video editing. In doing so, these studies mainly transfer the
inherent motion patterns from the source videos to the edited ones, where
results with inferior consistency to user prompts are often observed, due to
the lack of particular alignments between the delivered motions and edited
contents. To address this limitation, we present a shape-consistent video
editing method, namely StableV2V, in this paper. Our method decomposes the
entire editing pipeline into several sequential procedures, where it edits the
first video frame, then establishes an alignment between the delivered motions
and user prompts, and eventually propagates the edited contents to all other
frames based on such alignment. Furthermore, we curate a testing benchmark,
namely DAVIS-Edit, for a comprehensive evaluation of video editing, considering
various types of prompts and difficulties. Experimental results and analyses
illustrate the outperforming performance, visual consistency, and inference
efficiency of our method compared to existing state-of-the-art studies.
673c4bb46864fdcc715c8fd1
Chang Liu
claimed_verified
2024-11-19T13:09:10.693Z
6322cae4212b17c7728e7387
https://cdn-avatars.huggingface.co/v1/production/uploads/6322cae4212b17c7728e7387/p-hJt0795EO_wlUVY39rL.jpeg
Chang Liu
user
AlonzoLeeeooo
673c4bb46864fdcc715c8fd2
Rui Li
673c4bb46864fdcc715c8fd3
Kaidong Zhang
admin_assigned
2024-11-19T16:34:40.509Z
631c42f8aa346997917b562c
/avatars/900209c7c747914baf89175bcdb9f7c3.svg
Kaidong Zhang
user
hitachinsk
673c4bb46864fdcc715c8fd4
Yunwei Lan
673c4bb46864fdcc715c8fd5
Dong Liu
admin_assigned
2024-11-19T16:34:48.363Z
66b5dce0602c26aa8838ad3b
https://cdn-avatars.huggingface.co/v1/production/uploads/66b5dce0602c26aa8838ad3b/rcwOZ_yDyZxR54S5ftKoK.png
Dong Liu
user
DongLiu666
StableV2V: Stablizing Shape Consistency in Video-to-Video Editing
Recent advancements of generative AI have significantly promoted content
creation and editing, where prevailing studies further extend this exciting
progress to video editing. In doing so, these studies mainly transfer the
inherent motion patterns from the source videos to the edited ones, where
results with inferior consistency to user prompts are often observed, due to
the lack of particular alignments between the delivered motions and edited
contents. To address this limitation, we present a shape-consistent video
editing method, namely StableV2V, in this paper. Our method decomposes the
entire editing pipeline into several sequential procedures, where it edits the
first video frame, then establishes an alignment between the delivered motions
and user prompts, and eventually propagates the edited contents to all other
frames based on such alignment. Furthermore, we curate a testing benchmark,
namely DAVIS-Edit, for a comprehensive evaluation of video editing, considering
various types of prompts and difficulties. Experimental results and analyses
illustrate the outperforming performance, visual consistency, and inference
efficiency of our method compared to existing state-of-the-art studies.
StableV2V is a shape-consistent video editing method that decomposes the editing process into sequential steps to align motions with user prompts, outperforming existing methods in visual consistency and efficiency.
video editing shape-consistency alignment visual consistency inference efficiency DAVIS-Edit