title: ReVideo: Remake a Video with Motion and Content Control
summary: Despite significant advancements in video generation and editing using
  diffusion models, achieving accurate and localized video editing remains a
  substantial challenge. Additionally, most existing video editing methods
  primarily focus on altering visual content, with limited research dedicated to
  motion editing. In this paper, we present a novel attempt to Remake a Video
  (ReVideo) which stands out from existing methods by allowing precise video
  editing in specific areas through the specification of both content and motion.
  Content editing is facilitated by modifying the first frame, while the
  trajectory-based motion control offers an intuitive user interaction
  experience. ReVideo addresses a new task involving the coupling and training
  imbalance between content and motion control. To tackle this, we develop a
  three-stage training strategy that progressively decouples these two aspects
  from coarse to fine. Furthermore, we propose a spatiotemporal adaptive fusion
  module to integrate content and motion control across various sampling steps
  and spatial locations. Extensive experiments demonstrate that our ReVideo has
  promising performance on several accurate video editing applications, i.e., (1)
  locally changing video content while keeping the motion constant, (2) keeping
  content unchanged and customizing new motion trajectories, (3) modifying both
  content and motion trajectories. Our method can also seamlessly extend these
  applications to multi-area editing without specific training, demonstrating its
  flexibility and robustness.
paper_authors._id: 664ff85b7ef3fd6ce2fc0292
paper_authors.name: Chong Mou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-24T10:27:47.312Z
paper_authors._id: 63ec78f4c81b6a52391e1291
paper_authors.avatarUrl: /avatars/d13b35964c895f7aec69287390036a79.svg
paper_authors.fullname: ChongMou
paper_authors.type: user
paper_authors.user: Adapter
paper_authors._id: 664ff85b7ef3fd6ce2fc0293
paper_authors.name: Mingdeng Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-24T10:26:12.188Z
paper_authors._id: 6374a02d0856ac905bfc6113
paper_authors.avatarUrl: /avatars/2cbe75c9cc818a647ca6e416f129c96f.svg
paper_authors.fullname: Mingdeng Cao
paper_authors.type: user
paper_authors.user: Ljzycmd
paper_authors._id: 664ff85b7ef3fd6ce2fc0294
paper_authors.name: Xintao Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-24T10:26:18.902Z
paper_authors._id: 60e272ca6c78a8c122b12127
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60e272ca6c78a8c122b12127/xldEGBzGrU-bX6IwAw0Ie.jpeg
paper_authors.fullname: Xintao Wang
paper_authors.type: user
paper_authors.user: Xintao
paper_authors._id: 664ff85b7ef3fd6ce2fc0295
paper_authors.name: Zhaoyang Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-24T10:26:53.594Z
paper_authors._id: 658409ceca19ccf6d9989add
paper_authors.avatarUrl: /avatars/3ac1dbd25e52435185babdeb3da28875.svg
paper_authors.fullname: Zhaoyang Zhang
paper_authors.type: user
paper_authors.user: ZyZcuhk
paper_authors._id: 664ff85b7ef3fd6ce2fc0296
paper_authors.name: Ying Shan
paper_authors._id: 664ff85b7ef3fd6ce2fc0297
paper_authors.name: Jian Zhang
paper_title: ReVideo: Remake a Video with Motion and Content Control
paper_summary: Despite significant advancements in video generation and editing using
  diffusion models, achieving accurate and localized video editing remains a
  substantial challenge. Additionally, most existing video editing methods
  primarily focus on altering visual content, with limited research dedicated to
  motion editing. In this paper, we present a novel attempt to Remake a Video
  (ReVideo) which stands out from existing methods by allowing precise video
  editing in specific areas through the specification of both content and motion.
  Content editing is facilitated by modifying the first frame, while the
  trajectory-based motion control offers an intuitive user interaction
  experience. ReVideo addresses a new task involving the coupling and training
  imbalance between content and motion control. To tackle this, we develop a
  three-stage training strategy that progressively decouples these two aspects
  from coarse to fine. Furthermore, we propose a spatiotemporal adaptive fusion
  module to integrate content and motion control across various sampling steps
  and spatial locations. Extensive experiments demonstrate that our ReVideo has
  promising performance on several accurate video editing applications, i.e., (1)
  locally changing video content while keeping the motion constant, (2) keeping
  content unchanged and customizing new motion trajectories, (3) modifying both
  content and motion trajectories. Our method can also seamlessly extend these
  applications to multi-area editing without specific training, demonstrating its
  flexibility and robustness.
paper_ai_keywords: Remake a Video (ReVideo) diffusion models precise video editing content editing motion control trajectory-based motion control three-stage training strategy spatiotemporal adaptive fusion module locally changing video content new motion trajectories multi-area editing