title: MagicMotion: Controllable Video Generation with Dense-to-Sparse
    Trajectory Guidance
summary: Recent advances in video generation have led to remarkable improvements in
  visual quality and temporal coherence. Upon this, trajectory-controllable video
  generation has emerged to enable precise object motion control through
  explicitly defined spatial paths. However, existing methods struggle with
  complex object movements and multi-object motion control, resulting in
  imprecise trajectory adherence, poor object consistency, and compromised visual
  quality. Furthermore, these methods only support trajectory control in a single
  format, limiting their applicability in diverse scenarios. Additionally, there
  is no publicly available dataset or benchmark specifically tailored for
  trajectory-controllable video generation, hindering robust training and
  systematic evaluation. To address these challenges, we introduce MagicMotion, a
  novel image-to-video generation framework that enables trajectory control
  through three levels of conditions from dense to sparse: masks, bounding boxes,
  and sparse boxes. Given an input image and trajectories, MagicMotion seamlessly
  animates objects along defined trajectories while maintaining object
  consistency and visual quality. Furthermore, we present MagicData, a
  large-scale trajectory-controlled video dataset, along with an automated
  pipeline for annotation and filtering. We also introduce MagicBench, a
  comprehensive benchmark that assesses both video quality and trajectory control
  accuracy across different numbers of objects. Extensive experiments demonstrate
  that MagicMotion outperforms previous methods across various metrics. Our
  project page are publicly available at
  https://quanhaol.github.io/magicmotion-site.
paper_authors._id: 67dcd5913713a0e1da19bbe5
paper_authors.name: Quanhao Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-21T11:40:57.203Z
paper_authors._id: 65818917554131ec95df3829
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65818917554131ec95df3829/YIbgLCllm-0u88pRLhpVZ.jpeg
paper_authors.fullname: Quanhao Li
paper_authors.type: user
paper_authors.user: quanhaol
paper_authors._id: 67dcd5913713a0e1da19bbe6
paper_authors.name: Zhen Xing
paper_authors._id: 67dcd5913713a0e1da19bbe7
paper_authors.name: Rui Wang
paper_authors._id: 67dcd5913713a0e1da19bbe8
paper_authors.name: Hui Zhang
paper_authors._id: 67dcd5913713a0e1da19bbe9
paper_authors.name: Qi Dai
paper_authors._id: 67dcd5913713a0e1da19bbea
paper_authors.name: Zuxuan Wu
paper_title: MagicMotion: Controllable Video Generation with Dense-to-Sparse
    Trajectory Guidance
paper_summary: Recent advances in video generation have led to remarkable improvements in
  visual quality and temporal coherence. Upon this, trajectory-controllable video
  generation has emerged to enable precise object motion control through
  explicitly defined spatial paths. However, existing methods struggle with
  complex object movements and multi-object motion control, resulting in
  imprecise trajectory adherence, poor object consistency, and compromised visual
  quality. Furthermore, these methods only support trajectory control in a single
  format, limiting their applicability in diverse scenarios. Additionally, there
  is no publicly available dataset or benchmark specifically tailored for
  trajectory-controllable video generation, hindering robust training and
  systematic evaluation. To address these challenges, we introduce MagicMotion, a
  novel image-to-video generation framework that enables trajectory control
  through three levels of conditions from dense to sparse: masks, bounding boxes,
  and sparse boxes. Given an input image and trajectories, MagicMotion seamlessly
  animates objects along defined trajectories while maintaining object
  consistency and visual quality. Furthermore, we present MagicData, a
  large-scale trajectory-controlled video dataset, along with an automated
  pipeline for annotation and filtering. We also introduce MagicBench, a
  comprehensive benchmark that assesses both video quality and trajectory control
  accuracy across different numbers of objects. Extensive experiments demonstrate
  that MagicMotion outperforms previous methods across various metrics. Our
  project page are publicly available at
  https://quanhaol.github.io/magicmotion-site.
paper_ai_summary: MagicMotion is a trajectory-controllable video generation framework that supports dense, bounding box, and sparse box conditions, improving object consistency and visual quality, and is validated using the MagicData dataset and MagicBench benchmark.
paper_ai_keywords: trajectory-controllable video generation masks bounding boxes sparse boxes object consistency visual quality large-scale trajectory-controlled video dataset automated pipeline annotation filtering comprehensive benchmark video quality trajectory control accuracy