title: Image Conductor: Precision Control for Interactive Video Synthesis
summary: Filmmaking and animation production often require sophisticated techniques
  for coordinating camera transitions and object movements, typically involving
  labor-intensive real-world capturing. Despite advancements in generative AI for
  video creation, achieving precise control over motion for interactive video
  asset generation remains challenging. To this end, we propose Image Conductor,
  a method for precise control of camera transitions and object movements to
  generate video assets from a single image. An well-cultivated training strategy
  is proposed to separate distinct camera and object motion by camera LoRA
  weights and object LoRA weights. To further address cinematographic variations
  from ill-posed trajectories, we introduce a camera-free guidance technique
  during inference, enhancing object movements while eliminating camera
  transitions. Additionally, we develop a trajectory-oriented video motion data
  curation pipeline for training. Quantitative and qualitative experiments
  demonstrate our method's precision and fine-grained control in generating
  motion-controllable videos from images, advancing the practical application of
  interactive video synthesis. Project webpage available at
  https://liyaowei-stu.github.io/project/ImageConductor/
paper_authors._id: 667abc244ede40ecf16d00a9
paper_authors.name: Yaowei Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-26T13:16:27.685Z
paper_authors._id: 6362801380c1a705a6ea54ac
paper_authors.avatarUrl: /avatars/041ad5abf9be42e336938f51ebb8746c.svg
paper_authors.fullname: Yaowei Li
paper_authors.type: user
paper_authors.user: Yw22
paper_authors._id: 667abc244ede40ecf16d00aa
paper_authors.name: Xintao Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-26T13:16:34.274Z
paper_authors._id: 60e272ca6c78a8c122b12127
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60e272ca6c78a8c122b12127/xldEGBzGrU-bX6IwAw0Ie.jpeg
paper_authors.fullname: Xintao Wang
paper_authors.type: user
paper_authors.user: Xintao
paper_authors._id: 667abc244ede40ecf16d00ab
paper_authors.name: Zhaoyang Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-26T13:17:48.987Z
paper_authors._id: 658409ceca19ccf6d9989add
paper_authors.avatarUrl: /avatars/3ac1dbd25e52435185babdeb3da28875.svg
paper_authors.fullname: Zhaoyang Zhang
paper_authors.type: user
paper_authors.user: ZyZcuhk
paper_authors._id: 667abc244ede40ecf16d00ac
paper_authors.name: Zhouxia Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-26T13:18:00.332Z
paper_authors._id: 64db92a5858f8a41c11669b7
paper_authors.avatarUrl: /avatars/e834d8f1d4781e3bb0b5d6d25b3b3505.svg
paper_authors.fullname: Zhouxia Wang
paper_authors.type: user
paper_authors.user: wzhouxiff
paper_authors._id: 667abc244ede40ecf16d00ad
paper_authors.name: Ziyang Yuan
paper_authors._id: 667abc244ede40ecf16d00ae
paper_authors.name: Liangbin Xie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-26T13:18:14.101Z
paper_authors._id: 6315dcb493fdd4a5633f7b30
paper_authors.avatarUrl: /avatars/c58d228dbdf4f1edb5aa4e635dacb51d.svg
paper_authors.fullname: Liangbin Xie
paper_authors.type: user
paper_authors.user: Liangbin
paper_authors._id: 667abc244ede40ecf16d00af
paper_authors.name: Yuexian Zou
paper_authors._id: 667abc244ede40ecf16d00b0
paper_authors.name: Ying Shan
paper_title: Image Conductor: Precision Control for Interactive Video Synthesis
paper_summary: Filmmaking and animation production often require sophisticated techniques
  for coordinating camera transitions and object movements, typically involving
  labor-intensive real-world capturing. Despite advancements in generative AI for
  video creation, achieving precise control over motion for interactive video
  asset generation remains challenging. To this end, we propose Image Conductor,
  a method for precise control of camera transitions and object movements to
  generate video assets from a single image. An well-cultivated training strategy
  is proposed to separate distinct camera and object motion by camera LoRA
  weights and object LoRA weights. To further address cinematographic variations
  from ill-posed trajectories, we introduce a camera-free guidance technique
  during inference, enhancing object movements while eliminating camera
  transitions. Additionally, we develop a trajectory-oriented video motion data
  curation pipeline for training. Quantitative and qualitative experiments
  demonstrate our method's precision and fine-grained control in generating
  motion-controllable videos from images, advancing the practical application of
  interactive video synthesis. Project webpage available at
  https://liyaowei-stu.github.io/project/ImageConductor/
paper_ai_keywords: LoRA weights camera-free guidance technique trajectory-oriented video motion data curation pipeline