title: DragVideo: Interactive Drag-style Video Editing
summary: Editing visual content on videos remains a formidable challenge with two main
  issues: 1) direct and easy user control to produce 2) natural editing results
  without unsightly distortion and artifacts after changing shape, expression and
  layout. Inspired by DragGAN, a recent image-based drag-style editing technique,
  we address above issues by proposing DragVideo, where a similar drag-style user
  interaction is adopted to edit video content while maintaining temporal
  consistency. Empowered by recent diffusion models as in DragDiffusion,
  DragVideo contains the novel Drag-on-Video U-Net (DoVe) editing method, which
  optimizes diffused video latents generated by video U-Net to achieve the
  desired control. Specifically, we use Sample-specific LoRA fine-tuning and
  Mutual Self-Attention control to ensure faithful reconstruction of video from
  the DoVe method. We also present a series of testing examples for drag-style
  video editing and conduct extensive experiments across a wide array of
  challenging editing tasks, such as motion editing, skeleton editing, etc,
  underscoring DragVideo's versatility and generality. Our codes including the
  DragVideo web user interface will be released.
paper_authors._id: 656fe5ae34440bdf2eda5335
paper_authors.name: Yufan Deng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-06T13:27:12.313Z
paper_authors._id: 657073d2642d7f0ac42e3d6c
paper_authors.avatarUrl: /avatars/57cd9f4ca5590318042be8a0050485d7.svg
paper_authors.fullname: yfde
paper_authors.type: user
paper_authors.user: yfdeng
paper_authors._id: 656fe5ae34440bdf2eda5336
paper_authors.name: Ruida Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-10T15:12:47.192Z
paper_authors._id: 6618d1c3c1167c8d8702b19d
paper_authors.avatarUrl: /avatars/24611ca6c4158f4978ac8476a87d8d9c.svg
paper_authors.fullname: Ruida WANG
paper_authors.type: user
paper_authors.user: RickyDeSkywalker
paper_authors._id: 656fe5ae34440bdf2eda5337
paper_authors.name: Yuhao Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-06T12:44:47.502Z
paper_authors._id: 640ada00b871d7117d5ef66f
paper_authors.avatarUrl: /avatars/b45f369a370102ee338384c2154e41be.svg
paper_authors.fullname: ZHANG Yuaho
paper_authors.type: user
paper_authors.user: yzhanglp
paper_authors._id: 656fe5ae34440bdf2eda5338
paper_authors.name: Yu-Wing Tai
paper_authors._id: 656fe5ae34440bdf2eda5339
paper_authors.name: Chi-Keung Tang
paper_title: DragVideo: Interactive Drag-style Video Editing
paper_summary: Editing visual content on videos remains a formidable challenge with two main
  issues: 1) direct and easy user control to produce 2) natural editing results
  without unsightly distortion and artifacts after changing shape, expression and
  layout. Inspired by DragGAN, a recent image-based drag-style editing technique,
  we address above issues by proposing DragVideo, where a similar drag-style user
  interaction is adopted to edit video content while maintaining temporal
  consistency. Empowered by recent diffusion models as in DragDiffusion,
  DragVideo contains the novel Drag-on-Video U-Net (DoVe) editing method, which
  optimizes diffused video latents generated by video U-Net to achieve the
  desired control. Specifically, we use Sample-specific LoRA fine-tuning and
  Mutual Self-Attention control to ensure faithful reconstruction of video from
  the DoVe method. We also present a series of testing examples for drag-style
  video editing and conduct extensive experiments across a wide array of
  challenging editing tasks, such as motion editing, skeleton editing, etc,
  underscoring DragVideo's versatility and generality. Our codes including the
  DragVideo web user interface will be released.
paper_ai_summary: DragVideo, a drag-style video editing technique, uses diffusion models and U-Net architecture for natural and user-controlled editing while maintaining temporal consistency and versatility.
paper_ai_keywords: DragGAN drag-style editing diffusion models DragDiffusion Drag-on-Video U-Net DoVe Sample-specific LoRA fine-tuning Mutual Self-Attention control motion editing skeleton editing