title: TokenFlow: Consistent Diffusion Features for Consistent Video Editing
summary: The generative AI revolution has recently expanded to videos. Nevertheless,
  current state-of-the-art video models are still lagging behind image models in
  terms of visual quality and user control over the generated content. In this
  work, we present a framework that harnesses the power of a text-to-image
  diffusion model for the task of text-driven video editing. Specifically, given
  a source video and a target text-prompt, our method generates a high-quality
  video that adheres to the target text, while preserving the spatial layout and
  motion of the input video. Our method is based on a key observation that
  consistency in the edited video can be obtained by enforcing consistency in the
  diffusion feature space. We achieve this by explicitly propagating diffusion
  features based on inter-frame correspondences, readily available in the model.
  Thus, our framework does not require any training or fine-tuning, and can work
  in conjunction with any off-the-shelf text-to-image editing method. We
  demonstrate state-of-the-art editing results on a variety of real-world videos.
  Webpage: https://diffusion-tokenflow.github.io/
paper_authors._id: 64b9e3f6d4463c3d218bebe8
paper_authors.name: Michal Geyer
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-21T07:38:56.487Z
paper_authors._id: 6304fc4a34c824b17250a347
paper_authors.avatarUrl: /avatars/e2e2259a9cde8fd6981885892982f69e.svg
paper_authors.fullname: michal geyer
paper_authors.type: user
paper_authors.user: mishmish-g
paper_authors._id: 64b9e3f6d4463c3d218bebe9
paper_authors.name: Omer Bar-Tal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-21T07:39:12.500Z
paper_authors._id: 62e29044a133a252b5cf70b2
paper_authors.avatarUrl: /avatars/6d09ddcba9bc47c309150a8d77815891.svg
paper_authors.fullname: Omer Bar-Tal
paper_authors.type: user
paper_authors.user: omerbartal
paper_authors._id: 64b9e3f6d4463c3d218bebea
paper_authors.name: Shai Bagon
paper_authors._id: 64b9e3f6d4463c3d218bebeb
paper_authors.name: Tali Dekel
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-21T07:39:26.302Z
paper_authors._id: 631cddec68f7da9ad24f6fc7
paper_authors.avatarUrl: /avatars/7d4f1ce805e5889ca6594bd4a93f2583.svg
paper_authors.fullname: Tali Dekel
paper_authors.type: user
paper_authors.user: talidekel
paper_title: TokenFlow: Consistent Diffusion Features for Consistent Video Editing
paper_summary: The generative AI revolution has recently expanded to videos. Nevertheless,
  current state-of-the-art video models are still lagging behind image models in
  terms of visual quality and user control over the generated content. In this
  work, we present a framework that harnesses the power of a text-to-image
  diffusion model for the task of text-driven video editing. Specifically, given
  a source video and a target text-prompt, our method generates a high-quality
  video that adheres to the target text, while preserving the spatial layout and
  motion of the input video. Our method is based on a key observation that
  consistency in the edited video can be obtained by enforcing consistency in the
  diffusion feature space. We achieve this by explicitly propagating diffusion
  features based on inter-frame correspondences, readily available in the model.
  Thus, our framework does not require any training or fine-tuning, and can work
  in conjunction with any off-the-shelf text-to-image editing method. We
  demonstrate state-of-the-art editing results on a variety of real-world videos.
  Webpage: https://diffusion-tokenflow.github.io/
paper_ai_keywords: diffusion models text-to-image inter-frame correspondences diffusion feature space