VidEdit: Zero-Shot and Spatially Aware Text-Driven Video Editing
Recently, diffusion-based generative models have achieved remarkable success
for image generation and edition. However, their use for video editing still
faces important limitations. This paper introduces VidEdit, a novel method for
zero-shot text-based video editing ensuring strong temporal and spatial
consistency. Firstly, we propose to combine atlas-based and pre-trained
text-to-image diffusion models to provide a training-free and efficient editing
method, which by design fulfills temporal smoothness. Secondly, we leverage
off-the-shelf panoptic segmenters along with edge detectors and adapt their use
for conditioned diffusion-based atlas editing. This ensures a fine spatial
control on targeted regions while strictly preserving the structure of the
original video. Quantitative and qualitative experiments show that VidEdit
outperforms state-of-the-art methods on DAVIS dataset, regarding semantic
faithfulness, image preservation, and temporal consistency metrics. With this
framework, processing a single video only takes approximately one minute, and
it can generate multiple compatible edits based on a unique text prompt.
Project web-page at https://videdit.github.io
648beeec10bced401c50e780
Paul Couairon
claimed_verified
2025-06-16T09:53:18.403Z
653b7e9a7c01c693a19cf0ca
https://cdn-avatars.huggingface.co/v1/production/uploads/653b7e9a7c01c693a19cf0ca/J1w-Iu3qo6Ljko2rPjRMC.jpeg
Paul Couairon
user
paulcouairon
648beeec10bced401c50e781
Clément Rambour
648beeec10bced401c50e782
Jean-Emmanuel Haugeard
648beeec10bced401c50e783
Nicolas Thome
VidEdit: Zero-Shot and Spatially Aware Text-Driven Video Editing
Recently, diffusion-based generative models have achieved remarkable success
for image generation and edition. However, their use for video editing still
faces important limitations. This paper introduces VidEdit, a novel method for
zero-shot text-based video editing ensuring strong temporal and spatial
consistency. Firstly, we propose to combine atlas-based and pre-trained
text-to-image diffusion models to provide a training-free and efficient editing
method, which by design fulfills temporal smoothness. Secondly, we leverage
off-the-shelf panoptic segmenters along with edge detectors and adapt their use
for conditioned diffusion-based atlas editing. This ensures a fine spatial
control on targeted regions while strictly preserving the structure of the
original video. Quantitative and qualitative experiments show that VidEdit
outperforms state-of-the-art methods on DAVIS dataset, regarding semantic
faithfulness, image preservation, and temporal consistency metrics. With this
framework, processing a single video only takes approximately one minute, and
it can generate multiple compatible edits based on a unique text prompt.
Project web-page at https://videdit.github.io
VidEdit is a zero-shot text-based video editing method combining atlas-based and pre-trained diffusion models with panoptic segmenters and edge detectors to achieve strong temporal and spatial consistency.
diffusion-based generative models text-to-image diffusion models atlas-based editing panoptic segmenters edge detectors DAVIS dataset semantic faithfulness image preservation temporal consistency