title: I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion
    Models
summary: The remarkable generative capabilities of diffusion models have motivated
  extensive research in both image and video editing. Compared to video editing
  which faces additional challenges in the time dimension, image editing has
  witnessed the development of more diverse, high-quality approaches and more
  capable software like Photoshop. In light of this gap, we introduce a novel and
  generic solution that extends the applicability of image editing tools to
  videos by propagating edits from a single frame to the entire video using a
  pre-trained image-to-video model. Our method, dubbed I2VEdit, adaptively
  preserves the visual and motion integrity of the source video depending on the
  extent of the edits, effectively handling global edits, local edits, and
  moderate shape changes, which existing methods cannot fully achieve. At the
  core of our method are two main processes: Coarse Motion Extraction to align
  basic motion patterns with the original video, and Appearance Refinement for
  precise adjustments using fine-grained attention matching. We also incorporate
  a skip-interval strategy to mitigate quality degradation from auto-regressive
  generation across multiple video clips. Experimental results demonstrate our
  framework's superior performance in fine-grained video editing, proving its
  capability to produce high-quality, temporally consistent outputs.
paper_authors._id: 66555d9ca2d7a882a87cc9b0
paper_authors.name: Wenqi Ouyang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-28T10:31:30.172Z
paper_authors._id: 6487d9415d164d8c7f905bcf
paper_authors.avatarUrl: /avatars/1c01dd4b88f88a9863a12c09a9a28bed.svg
paper_authors.fullname: Wenqi Ouyang
paper_authors.type: user
paper_authors.user: Vicky0522
paper_authors._id: 66555d9ca2d7a882a87cc9b1
paper_authors.name: Yi Dong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-05-28T15:21:39.190Z
paper_authors._id: 661f639f7273a6cb22767c14
paper_authors.avatarUrl: /avatars/8b65440edf21e1fd98c90e2776bd8e0c.svg
paper_authors.fullname: Dong Yi
paper_authors.type: user
paper_authors.user: ntuyi
paper_authors._id: 66555d9ca2d7a882a87cc9b2
paper_authors.name: Lei Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-10T08:09:53.918Z
paper_authors._id: 6626a471430a124253f197c8
paper_authors.avatarUrl: /avatars/f5747fdbe495d1296fed9d16d8c95857.svg
paper_authors.fullname: yl-1993
paper_authors.type: user
paper_authors.user: yl-1993
paper_authors._id: 66555d9ca2d7a882a87cc9b3
paper_authors.name: Jianlou Si
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-28T10:30:41.704Z
paper_authors._id: 6409693abf3e9a4bb2e64dfd
paper_authors.avatarUrl: /avatars/9ddeceec9f4dae3a813165016eb6f4d3.svg
paper_authors.fullname: Jianlou
paper_authors.type: user
paper_authors.user: Jianlou
paper_authors._id: 66555d9ca2d7a882a87cc9b4
paper_authors.name: Xingang Pan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-28T10:30:21.693Z
paper_authors._id: 646758d6a9b4610868a138ff
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/646758d6a9b4610868a138ff/fNBlK5g4g7vlPAR1bQimZ.png
paper_authors.fullname: Xingang Pan
paper_authors.type: user
paper_authors.user: Xingang-Pan
paper_title: I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion
    Models
paper_summary: The remarkable generative capabilities of diffusion models have motivated
  extensive research in both image and video editing. Compared to video editing
  which faces additional challenges in the time dimension, image editing has
  witnessed the development of more diverse, high-quality approaches and more
  capable software like Photoshop. In light of this gap, we introduce a novel and
  generic solution that extends the applicability of image editing tools to
  videos by propagating edits from a single frame to the entire video using a
  pre-trained image-to-video model. Our method, dubbed I2VEdit, adaptively
  preserves the visual and motion integrity of the source video depending on the
  extent of the edits, effectively handling global edits, local edits, and
  moderate shape changes, which existing methods cannot fully achieve. At the
  core of our method are two main processes: Coarse Motion Extraction to align
  basic motion patterns with the original video, and Appearance Refinement for
  precise adjustments using fine-grained attention matching. We also incorporate
  a skip-interval strategy to mitigate quality degradation from auto-regressive
  generation across multiple video clips. Experimental results demonstrate our
  framework's superior performance in fine-grained video editing, proving its
  capability to produce high-quality, temporally consistent outputs.
paper_ai_keywords: diffusion models I2VEdit Coarse Motion Extraction Appearance Refinement fine-grained attention matching skip-interval strategy auto-regressive generation