title: ObjCtrl-2.5D: Training-free Object Control with Camera Poses
summary: This study aims to achieve more precise and versatile object control in
  image-to-video (I2V) generation. Current methods typically represent the
  spatial movement of target objects with 2D trajectories, which often fail to
  capture user intention and frequently produce unnatural results. To enhance
  control, we present ObjCtrl-2.5D, a training-free object control approach that
  uses a 3D trajectory, extended from a 2D trajectory with depth information, as
  a control signal. By modeling object movement as camera movement, ObjCtrl-2.5D
  represents the 3D trajectory as a sequence of camera poses, enabling object
  motion control using an existing camera motion control I2V generation model
  (CMC-I2V) without training. To adapt the CMC-I2V model originally designed for
  global motion control to handle local object motion, we introduce a module to
  isolate the target object from the background, enabling independent local
  control. In addition, we devise an effective way to achieve more accurate
  object control by sharing low-frequency warped latent within the object's
  region across frames. Extensive experiments demonstrate that ObjCtrl-2.5D
  significantly improves object control accuracy compared to training-free
  methods and offers more diverse control capabilities than training-based
  approaches using 2D trajectories, enabling complex effects like object
  rotation. Code and results are available at
  https://wzhouxiff.github.io/projects/ObjCtrl-2.5D/.
paper_authors._id: 6759180441b35c9329025a5e
paper_authors.name: Zhouxia Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-11T13:50:16.745Z
paper_authors._id: 64db92a5858f8a41c11669b7
paper_authors.avatarUrl: /avatars/e834d8f1d4781e3bb0b5d6d25b3b3505.svg
paper_authors.fullname: Zhouxia Wang
paper_authors.type: user
paper_authors.user: wzhouxiff
paper_authors._id: 6759180441b35c9329025a5f
paper_authors.name: Yushi Lan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-11T13:50:10.996Z
paper_authors._id: 64b55f62e6b0c53f97e64034
paper_authors.avatarUrl: /avatars/d5fc53de37d4fb0b5095a5fb21b43503.svg
paper_authors.fullname: LAN YUSHI
paper_authors.type: user
paper_authors.user: yslan
paper_authors._id: 6759180441b35c9329025a60
paper_authors.name: Shangchen Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-11T13:50:00.183Z
paper_authors._id: 62e57662ae9d3f10acbb1b1b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62e57662ae9d3f10acbb1b1b/lg58jdbNyv6LGH2LFnZDF.png
paper_authors.fullname: Shangchen Zhou
paper_authors.type: user
paper_authors.user: sczhou
paper_authors._id: 6759180441b35c9329025a61
paper_authors.name: Chen Change Loy
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-11T13:49:54.135Z
paper_authors._id: 67459d997a49660f7f62452f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/X25yTdxC_hcLVxpx4voZo.png
paper_authors.fullname: Chen Change Loy
paper_authors.type: user
paper_authors.user: cavanloy
paper_title: ObjCtrl-2.5D: Training-free Object Control with Camera Poses
paper_summary: This study aims to achieve more precise and versatile object control in
  image-to-video (I2V) generation. Current methods typically represent the
  spatial movement of target objects with 2D trajectories, which often fail to
  capture user intention and frequently produce unnatural results. To enhance
  control, we present ObjCtrl-2.5D, a training-free object control approach that
  uses a 3D trajectory, extended from a 2D trajectory with depth information, as
  a control signal. By modeling object movement as camera movement, ObjCtrl-2.5D
  represents the 3D trajectory as a sequence of camera poses, enabling object
  motion control using an existing camera motion control I2V generation model
  (CMC-I2V) without training. To adapt the CMC-I2V model originally designed for
  global motion control to handle local object motion, we introduce a module to
  isolate the target object from the background, enabling independent local
  control. In addition, we devise an effective way to achieve more accurate
  object control by sharing low-frequency warped latent within the object's
  region across frames. Extensive experiments demonstrate that ObjCtrl-2.5D
  significantly improves object control accuracy compared to training-free
  methods and offers more diverse control capabilities than training-based
  approaches using 2D trajectories, enabling complex effects like object
  rotation. Code and results are available at
  https://wzhouxiff.github.io/projects/ObjCtrl-2.5D/.
paper_ai_keywords: ObjCtrl-2.5D 3D trajectory 2D trajectory depth information camera movement CMC-I2V global motion control local object motion module isolate background independent local control low-frequency warped latent object control accuracy complex effects object rotation