title: Trajectory Attention for Fine-grained Video Motion Control
summary: Recent advancements in video generation have been greatly driven by video
  diffusion models, with camera motion control emerging as a crucial challenge in
  creating view-customized visual content. This paper introduces trajectory
  attention, a novel approach that performs attention along available pixel
  trajectories for fine-grained camera motion control. Unlike existing methods
  that often yield imprecise outputs or neglect temporal correlations, our
  approach possesses a stronger inductive bias that seamlessly injects trajectory
  information into the video generation process. Importantly, our approach models
  trajectory attention as an auxiliary branch alongside traditional temporal
  attention. This design enables the original temporal attention and the
  trajectory attention to work in synergy, ensuring both precise motion control
  and new content generation capability, which is critical when the trajectory is
  only partially available. Experiments on camera motion control for images and
  videos demonstrate significant improvements in precision and long-range
  consistency while maintaining high-quality generation. Furthermore, we show
  that our approach can be extended to other video motion control tasks, such as
  first-frame-guided video editing, where it excels in maintaining content
  consistency over large spatial and temporal ranges.
paper_authors._id: 674d3226810801f2fa57b8ed
paper_authors.name: Zeqi Xiao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-02T09:39:04.354Z
paper_authors._id: 650e37cc11f3210cf7910501
paper_authors.avatarUrl: /avatars/dab5c9d647cfa97c59f5170216673a20.svg
paper_authors.fullname: zeqixiao
paper_authors.type: user
paper_authors.user: zeqixiao
paper_authors._id: 674d3226810801f2fa57b8ee
paper_authors.name: Wenqi Ouyang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-02T09:38:53.298Z
paper_authors._id: 6487d9415d164d8c7f905bcf
paper_authors.avatarUrl: /avatars/1c01dd4b88f88a9863a12c09a9a28bed.svg
paper_authors.fullname: Wenqi Ouyang
paper_authors.type: user
paper_authors.user: Vicky0522
paper_authors._id: 674d3226810801f2fa57b8ef
paper_authors.name: Yifan Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-02T12:52:33.458Z
paper_authors._id: 64a63f9449b08110f761cd73
paper_authors.avatarUrl: /avatars/61860202fc818b105ef24e74dd4f7d3c.svg
paper_authors.fullname: Yifan Zhou
paper_authors.type: user
paper_authors.user: SingleZombie
paper_authors._id: 674d3226810801f2fa57b8f0
paper_authors.name: Shuai Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-02T09:38:33.890Z
paper_authors._id: 62a54d0410334c1d024e2f59
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1664764278226-62a54d0410334c1d024e2f59.jpeg
paper_authors.fullname: Shuai Yang
paper_authors.type: user
paper_authors.user: PKUWilliamYang
paper_authors._id: 674d3226810801f2fa57b8f1
paper_authors.name: Lei Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-02T12:52:31.720Z
paper_authors._id: 6626a471430a124253f197c8
paper_authors.avatarUrl: /avatars/f5747fdbe495d1296fed9d16d8c95857.svg
paper_authors.fullname: yl-1993
paper_authors.type: user
paper_authors.user: yl-1993
paper_authors._id: 674d3226810801f2fa57b8f2
paper_authors.name: Jianlou Si
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-02T09:38:05.905Z
paper_authors._id: 6409693abf3e9a4bb2e64dfd
paper_authors.avatarUrl: /avatars/9ddeceec9f4dae3a813165016eb6f4d3.svg
paper_authors.fullname: Jianlou
paper_authors.type: user
paper_authors.user: Jianlou
paper_authors._id: 674d3226810801f2fa57b8f3
paper_authors.name: Xingang Pan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-02T09:37:47.316Z
paper_authors._id: 646758d6a9b4610868a138ff
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/646758d6a9b4610868a138ff/fNBlK5g4g7vlPAR1bQimZ.png
paper_authors.fullname: Xingang Pan
paper_authors.type: user
paper_authors.user: Xingang-Pan
paper_title: Trajectory Attention for Fine-grained Video Motion Control
paper_summary: Recent advancements in video generation have been greatly driven by video
  diffusion models, with camera motion control emerging as a crucial challenge in
  creating view-customized visual content. This paper introduces trajectory
  attention, a novel approach that performs attention along available pixel
  trajectories for fine-grained camera motion control. Unlike existing methods
  that often yield imprecise outputs or neglect temporal correlations, our
  approach possesses a stronger inductive bias that seamlessly injects trajectory
  information into the video generation process. Importantly, our approach models
  trajectory attention as an auxiliary branch alongside traditional temporal
  attention. This design enables the original temporal attention and the
  trajectory attention to work in synergy, ensuring both precise motion control
  and new content generation capability, which is critical when the trajectory is
  only partially available. Experiments on camera motion control for images and
  videos demonstrate significant improvements in precision and long-range
  consistency while maintaining high-quality generation. Furthermore, we show
  that our approach can be extended to other video motion control tasks, such as
  first-frame-guided video editing, where it excels in maintaining content
  consistency over large spatial and temporal ranges.
paper_ai_summary: Trajectory attention is introduced to enhance video generation by incorporating motion control along pixel trajectories, improving precision and consistency.
paper_ai_keywords: video diffusion models camera motion control trajectory attention temporal attention first-frame-guided video editing