MagicProp: Diffusion-based Video Editing via Motion-aware Appearance
  Propagation
This paper addresses the issue of modifying the visual appearance of videos
while preserving their motion. A novel framework, named MagicProp, is proposed,
which disentangles the video editing process into two stages: appearance
editing and motion-aware appearance propagation. In the first stage, MagicProp
selects a single frame from the input video and applies image-editing
techniques to modify the content and/or style of the frame. The flexibility of
these techniques enables the editing of arbitrary regions within the frame. In
the second stage, MagicProp employs the edited frame as an appearance reference
and generates the remaining frames using an autoregressive rendering approach.
To achieve this, a diffusion-based conditional generation model, called
PropDPM, is developed, which synthesizes the target frame by conditioning on
the reference appearance, the target motion, and its previous appearance. The
autoregressive editing approach ensures temporal consistency in the resulting
videos. Overall, MagicProp combines the flexibility of image-editing techniques
with the superior temporal consistency of autoregressive modeling, enabling
flexible editing of object types and aesthetic styles in arbitrary regions of
input videos while maintaining good temporal consistency across frames.
Extensive experiments in various video editing scenarios demonstrate the
effectiveness of MagicProp.
64f829718a234f114e1d42e3
Hanshu Yan
claimed_verified
2023-12-22T07:37:40.307Z
631eec57daa9591e522ee721
/avatars/39a6fd3b9717c142b73be8c81c69ebe5.svg
PeRFlow
user
hansyan
64f829718a234f114e1d42e4
Jun Hao Liew
admin_assigned
2023-09-06T15:26:00.694Z
6305bb149d2531fabd198638
/avatars/0ec3c55d445264d43c0430f9edf88bf8.svg
Jun Hao Liew
user
junhao910323
64f829718a234f114e1d42e5
Long Mai
admin_assigned
2023-09-06T15:26:14.993Z
630d65363dc31beba6f3cc78
/avatars/f0f19fb134f4f4cb5e64573978993dd6.svg
Long Mai
user
mtlong
64f829718a234f114e1d42e6
Shanchuan Lin
claimed_verified
2023-12-05T08:41:48.755Z
645863f7dc18eb1a9b5d29df
https://cdn-avatars.huggingface.co/v1/production/uploads/645863f7dc18eb1a9b5d29df/t49Nnyl4tbkUn7CmQqKZh.jpeg
Peter Lin
user
PeterL1n
64f829718a234f114e1d42e7
Jiashi Feng
MagicProp: Diffusion-based Video Editing via Motion-aware Appearance
  Propagation
This paper addresses the issue of modifying the visual appearance of videos
while preserving their motion. A novel framework, named MagicProp, is proposed,
which disentangles the video editing process into two stages: appearance
editing and motion-aware appearance propagation. In the first stage, MagicProp
selects a single frame from the input video and applies image-editing
techniques to modify the content and/or style of the frame. The flexibility of
these techniques enables the editing of arbitrary regions within the frame. In
the second stage, MagicProp employs the edited frame as an appearance reference
and generates the remaining frames using an autoregressive rendering approach.
To achieve this, a diffusion-based conditional generation model, called
PropDPM, is developed, which synthesizes the target frame by conditioning on
the reference appearance, the target motion, and its previous appearance. The
autoregressive editing approach ensures temporal consistency in the resulting
videos. Overall, MagicProp combines the flexibility of image-editing techniques
with the superior temporal consistency of autoregressive modeling, enabling
flexible editing of object types and aesthetic styles in arbitrary regions of
input videos while maintaining good temporal consistency across frames.
Extensive experiments in various video editing scenarios demonstrate the
effectiveness of MagicProp.
MagicProp combines image editing and motion-aware appearance propagation using a diffusion-based model to modify video appearance while preserving motion consistency.
appearance editing motion-aware appearance propagation image-editing techniques autoregressive rendering approach diffusion-based conditional generation model PropDPM temporal consistency