title: VideoSwap: Customized Video Subject Swapping with Interactive Semantic
    Point Correspondence
summary: Current diffusion-based video editing primarily focuses on
  structure-preserved editing by utilizing various dense correspondences to
  ensure temporal consistency and motion alignment. However, these approaches are
  often ineffective when the target edit involves a shape change. To embark on
  video editing with shape change, we explore customized video subject swapping
  in this work, where we aim to replace the main subject in a source video with a
  target subject having a distinct identity and potentially different shape. In
  contrast to previous methods that rely on dense correspondences, we introduce
  the VideoSwap framework that exploits semantic point correspondences, inspired
  by our observation that only a small number of semantic points are necessary to
  align the subject's motion trajectory and modify its shape. We also introduce
  various user-point interactions (\eg, removing points and dragging points) to
  address various semantic point correspondence. Extensive experiments
  demonstrate state-of-the-art video subject swapping results across a variety of
  real-world videos.
paper_authors._id: 656eb2b28e7f2775acfc13d8
paper_authors.name: Yuchao Gu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-05T09:25:39.377Z
paper_authors._id: 63021630a35b21bd8a53305a
paper_authors.avatarUrl: /avatars/7a7e8b39749eda61e57d8a1908726558.svg
paper_authors.fullname: Gu Yuchao
paper_authors.type: user
paper_authors.user: guyuchao
paper_authors._id: 656eb2b28e7f2775acfc13d9
paper_authors.name: Yipin Zhou
paper_authors._id: 656eb2b28e7f2775acfc13da
paper_authors.name: Bichen Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-05T09:26:04.012Z
paper_authors._id: 6363749520c2cf37bccd9e84
paper_authors.avatarUrl: /avatars/e9de7911e67090d883f15a7a58f18e4e.svg
paper_authors.fullname: Bichen Wu
paper_authors.type: user
paper_authors.user: bichenwu
paper_authors._id: 656eb2b28e7f2775acfc13db
paper_authors.name: Licheng Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-05T09:26:11.614Z
paper_authors._id: 6372b1f9bd81fae2b3a712e8
paper_authors.avatarUrl: /avatars/bcdf7b540cc94882e0ae6035bb07a8f2.svg
paper_authors.fullname: Licheng Yu
paper_authors.type: user
paper_authors.user: lichengyu
paper_authors._id: 656eb2b28e7f2775acfc13dc
paper_authors.name: Jia-Wei Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-05T09:26:35.038Z
paper_authors._id: 64230048a73327caad9d8241
paper_authors.avatarUrl: /avatars/ac115f1a21c743db7c925f0e18451145.svg
paper_authors.fullname: Jiawei Liu
paper_authors.type: user
paper_authors.user: JiaweiLIU
paper_authors._id: 656eb2b28e7f2775acfc13dd
paper_authors.name: Rui Zhao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-05T09:26:59.193Z
paper_authors._id: 652b83b73b5997ed71a310f2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/652b83b73b5997ed71a310f2/ipCpdeHUp4-0OmRz5z8IW.png
paper_authors.fullname: Rui Zhao
paper_authors.type: user
paper_authors.user: ruizhaocv
paper_authors._id: 656eb2b28e7f2775acfc13de
paper_authors.name: Jay Zhangjie Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-06T10:25:09.481Z
paper_authors._id: 633aaf695df91da9cea92960
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/633aaf695df91da9cea92960/h1oOWUudbdlsd63Q5w0hM.png
paper_authors.fullname: Jay Wu
paper_authors.type: user
paper_authors.user: jayw
paper_authors._id: 656eb2b28e7f2775acfc13df
paper_authors.name: David Junhao Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-05T09:27:34.361Z
paper_authors._id: 6468855763a564ba347e1efa
paper_authors.avatarUrl: /avatars/caa99c3318c1d6d21be44d2d32795a62.svg
paper_authors.fullname: David Junhao ZHANG
paper_authors.type: user
paper_authors.user: Junhao233
paper_authors._id: 656eb2b28e7f2775acfc13e0
paper_authors.name: Mike Zheng Shou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-05T09:28:06.371Z
paper_authors._id: 63a55320ce5763e06f78519c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1671779060549-noauth.jpeg
paper_authors.fullname: Mike Shou
paper_authors.type: user
paper_authors.user: mikeshou
paper_authors._id: 656eb2b28e7f2775acfc13e1
paper_authors.name: Kevin Tang
paper_title: VideoSwap: Customized Video Subject Swapping with Interactive Semantic
    Point Correspondence
paper_summary: Current diffusion-based video editing primarily focuses on
  structure-preserved editing by utilizing various dense correspondences to
  ensure temporal consistency and motion alignment. However, these approaches are
  often ineffective when the target edit involves a shape change. To embark on
  video editing with shape change, we explore customized video subject swapping
  in this work, where we aim to replace the main subject in a source video with a
  target subject having a distinct identity and potentially different shape. In
  contrast to previous methods that rely on dense correspondences, we introduce
  the VideoSwap framework that exploits semantic point correspondences, inspired
  by our observation that only a small number of semantic points are necessary to
  align the subject's motion trajectory and modify its shape. We also introduce
  various user-point interactions (\eg, removing points and dragging points) to
  address various semantic point correspondence. Extensive experiments
  demonstrate state-of-the-art video subject swapping results across a variety of
  real-world videos.
paper_ai_summary: The VideoSwap framework uses semantic point correspondences for video subject swapping, achieving superior results in handling shape changes compared to dense correspondence methods.
paper_ai_keywords: diffusion-based video editing structure-preserved editing dense correspondences VideoSwap semantic point correspondences user-point interactions