title: LeviTor: 3D Trajectory Oriented Image-to-Video Synthesis
summary: The intuitive nature of drag-based interaction has led to its growing
  adoption for controlling object trajectories in image-to-video synthesis.
  Still, existing methods that perform dragging in the 2D space usually face
  ambiguity when handling out-of-plane movements. In this work, we augment the
  interaction with a new dimension, i.e., the depth dimension, such that users
  are allowed to assign a relative depth for each point on the trajectory. That
  way, our new interaction paradigm not only inherits the convenience from 2D
  dragging, but facilitates trajectory control in the 3D space, broadening the
  scope of creativity. We propose a pioneering method for 3D trajectory control
  in image-to-video synthesis by abstracting object masks into a few cluster
  points. These points, accompanied by the depth information and the instance
  information, are finally fed into a video diffusion model as the control
  signal. Extensive experiments validate the effectiveness of our approach,
  dubbed LeviTor, in precisely manipulating the object movements when producing
  photo-realistic videos from static images. Project page:
  https://ppetrichor.github.io/levitor.github.io/
paper_authors._id: 6764f548cee1fdbd9765e9bc
paper_authors.name: Hanlin Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-20T08:32:36.964Z
paper_authors._id: 665f059a8947302aa2c63afe
paper_authors.avatarUrl: /avatars/50f560285946532321a0bd526494148d.svg
paper_authors.fullname: hanlin wang
paper_authors.type: user
paper_authors.user: hlwang06
paper_authors._id: 6764f548cee1fdbd9765e9bd
paper_authors.name: Hao Ouyang
paper_authors._id: 6764f548cee1fdbd9765e9be
paper_authors.name: Qiuyu Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-20T08:32:28.937Z
paper_authors._id: 64981bea09cea550852652af
paper_authors.avatarUrl: /avatars/df528e9008972c8e5ae4d278e617476c.svg
paper_authors.fullname: Qiuyu Wang
paper_authors.type: user
paper_authors.user: qiuyuu
paper_authors._id: 6764f548cee1fdbd9765e9bf
paper_authors.name: Wen Wang
paper_authors._id: 6764f548cee1fdbd9765e9c0
paper_authors.name: Ka Leong Cheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-20T09:26:43.630Z
paper_authors._id: 64acd2ec39fcfebff8c79c00
paper_authors.avatarUrl: /avatars/9419384846b92182f2c47ce2fbd0f8d3.svg
paper_authors.fullname: Ka Leong Cheng
paper_authors.type: user
paper_authors.user: felixcheng97
paper_authors._id: 6764f548cee1fdbd9765e9c1
paper_authors.name: Qifeng Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-20T09:26:37.950Z
paper_authors._id: 6467b121e7a6a374fd19b44b
paper_authors.avatarUrl: /avatars/3f2874d58986d651aef55e3408b05700.svg
paper_authors.fullname: Qifeng Chen
paper_authors.type: user
paper_authors.user: cqf
paper_authors._id: 6764f548cee1fdbd9765e9c2
paper_authors.name: Yujun Shen
paper_authors._id: 6764f548cee1fdbd9765e9c3
paper_authors.name: Limin Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-20T09:26:23.360Z
paper_authors._id: 62c77f4352d8ae531f5511f9
paper_authors.avatarUrl: /avatars/50198ccb02ccd286975a4613fbabee28.svg
paper_authors.fullname: Limin Wang
paper_authors.type: user
paper_authors.user: lmwang
paper_title: LeviTor: 3D Trajectory Oriented Image-to-Video Synthesis
paper_summary: The intuitive nature of drag-based interaction has led to its growing
  adoption for controlling object trajectories in image-to-video synthesis.
  Still, existing methods that perform dragging in the 2D space usually face
  ambiguity when handling out-of-plane movements. In this work, we augment the
  interaction with a new dimension, i.e., the depth dimension, such that users
  are allowed to assign a relative depth for each point on the trajectory. That
  way, our new interaction paradigm not only inherits the convenience from 2D
  dragging, but facilitates trajectory control in the 3D space, broadening the
  scope of creativity. We propose a pioneering method for 3D trajectory control
  in image-to-video synthesis by abstracting object masks into a few cluster
  points. These points, accompanied by the depth information and the instance
  information, are finally fed into a video diffusion model as the control
  signal. Extensive experiments validate the effectiveness of our approach,
  dubbed LeviTor, in precisely manipulating the object movements when producing
  photo-realistic videos from static images. Project page:
  https://ppetrichor.github.io/levitor.github.io/
paper_ai_summary: A 3D drag-based interaction method that uses object masks and depth information to control trajectories in image-to-video synthesis, enhancing creativity and precision.
paper_ai_keywords: drag-based interaction 3D trajectory control image-to-video synthesis object masks cluster points depth information instance information video diffusion model LeviTor