title: MagicProp: Diffusion-based Video Editing via Motion-aware Appearance
    Propagation
summary: This paper addresses the issue of modifying the visual appearance of videos
  while preserving their motion. A novel framework, named MagicProp, is proposed,
  which disentangles the video editing process into two stages: appearance
  editing and motion-aware appearance propagation. In the first stage, MagicProp
  selects a single frame from the input video and applies image-editing
  techniques to modify the content and/or style of the frame. The flexibility of
  these techniques enables the editing of arbitrary regions within the frame. In
  the second stage, MagicProp employs the edited frame as an appearance reference
  and generates the remaining frames using an autoregressive rendering approach.
  To achieve this, a diffusion-based conditional generation model, called
  PropDPM, is developed, which synthesizes the target frame by conditioning on
  the reference appearance, the target motion, and its previous appearance. The
  autoregressive editing approach ensures temporal consistency in the resulting
  videos. Overall, MagicProp combines the flexibility of image-editing techniques
  with the superior temporal consistency of autoregressive modeling, enabling
  flexible editing of object types and aesthetic styles in arbitrary regions of
  input videos while maintaining good temporal consistency across frames.
  Extensive experiments in various video editing scenarios demonstrate the
  effectiveness of MagicProp.
paper_authors._id: 64f829718a234f114e1d42e3
paper_authors.name: Hanshu Yan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-22T07:37:40.307Z
paper_authors._id: 631eec57daa9591e522ee721
paper_authors.avatarUrl: /avatars/39a6fd3b9717c142b73be8c81c69ebe5.svg
paper_authors.fullname: PeRFlow
paper_authors.type: user
paper_authors.user: hansyan
paper_authors._id: 64f829718a234f114e1d42e4
paper_authors.name: Jun Hao Liew
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-06T15:26:00.694Z
paper_authors._id: 6305bb149d2531fabd198638
paper_authors.avatarUrl: /avatars/0ec3c55d445264d43c0430f9edf88bf8.svg
paper_authors.fullname: Jun Hao Liew
paper_authors.type: user
paper_authors.user: junhao910323
paper_authors._id: 64f829718a234f114e1d42e5
paper_authors.name: Long Mai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-06T15:26:14.993Z
paper_authors._id: 630d65363dc31beba6f3cc78
paper_authors.avatarUrl: /avatars/f0f19fb134f4f4cb5e64573978993dd6.svg
paper_authors.fullname: Long Mai
paper_authors.type: user
paper_authors.user: mtlong
paper_authors._id: 64f829718a234f114e1d42e6
paper_authors.name: Shanchuan Lin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-05T08:41:48.755Z
paper_authors._id: 645863f7dc18eb1a9b5d29df
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/645863f7dc18eb1a9b5d29df/t49Nnyl4tbkUn7CmQqKZh.jpeg
paper_authors.fullname: Peter Lin
paper_authors.type: user
paper_authors.user: PeterL1n
paper_authors._id: 64f829718a234f114e1d42e7
paper_authors.name: Jiashi Feng
paper_title: MagicProp: Diffusion-based Video Editing via Motion-aware Appearance
    Propagation
paper_summary: This paper addresses the issue of modifying the visual appearance of videos
  while preserving their motion. A novel framework, named MagicProp, is proposed,
  which disentangles the video editing process into two stages: appearance
  editing and motion-aware appearance propagation. In the first stage, MagicProp
  selects a single frame from the input video and applies image-editing
  techniques to modify the content and/or style of the frame. The flexibility of
  these techniques enables the editing of arbitrary regions within the frame. In
  the second stage, MagicProp employs the edited frame as an appearance reference
  and generates the remaining frames using an autoregressive rendering approach.
  To achieve this, a diffusion-based conditional generation model, called
  PropDPM, is developed, which synthesizes the target frame by conditioning on
  the reference appearance, the target motion, and its previous appearance. The
  autoregressive editing approach ensures temporal consistency in the resulting
  videos. Overall, MagicProp combines the flexibility of image-editing techniques
  with the superior temporal consistency of autoregressive modeling, enabling
  flexible editing of object types and aesthetic styles in arbitrary regions of
  input videos while maintaining good temporal consistency across frames.
  Extensive experiments in various video editing scenarios demonstrate the
  effectiveness of MagicProp.
paper_ai_keywords: diffusion-based conditional generation model PropDPM autoregressive rendering autoregressive editing temporal consistency disentangles the video editing process appearance editing motion-aware appearance propagation diffusion-based model single frame selection image-editing techniques arbitrary regions video editing scenarios flexibility of image-editing techniques conditional generation