title: Direct-a-Video: Customized Video Generation with User-Directed Camera
    Movement and Object Motion
summary: Recent text-to-video diffusion models have achieved impressive progress. In
  practice, users often desire the ability to control object motion and camera
  movement independently for customized video creation. However, current methods
  lack the focus on separately controlling object motion and camera movement in a
  decoupled manner, which limits the controllability and flexibility of
  text-to-video models. In this paper, we introduce Direct-a-Video, a system that
  allows users to independently specify motions for one or multiple objects
  and/or camera movements, as if directing a video. We propose a simple yet
  effective strategy for the decoupled control of object motion and camera
  movement. Object motion is controlled through spatial cross-attention
  modulation using the model's inherent priors, requiring no additional
  optimization. For camera movement, we introduce new temporal cross-attention
  layers to interpret quantitative camera movement parameters. We further employ
  an augmentation-based approach to train these layers in a self-supervised
  manner on a small-scale dataset, eliminating the need for explicit motion
  annotation. Both components operate independently, allowing individual or
  combined control, and can generalize to open-domain scenarios. Extensive
  experiments demonstrate the superiority and effectiveness of our method.
  Project page: https://direct-a-video.github.io/.
paper_authors._id: 65c1a29532ff878c506343a7
paper_authors.name: Shiyuan Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-17T14:43:36.241Z
paper_authors._id: 63316d499e3604f3f17f5d89
paper_authors.avatarUrl: /avatars/c7e0c8bdd7e598276cfd133d5222c5e8.svg
paper_authors.fullname: catfood
paper_authors.type: user
paper_authors.user: ysy31415926
paper_authors._id: 65c1a29532ff878c506343a8
paper_authors.name: Liang Hou
paper_authors._id: 65c1a29532ff878c506343a9
paper_authors.name: Haibin Huang
paper_authors._id: 65c1a29532ff878c506343aa
paper_authors.name: Chongyang Ma
paper_authors._id: 65c1a29532ff878c506343ab
paper_authors.name: Pengfei Wan
paper_authors._id: 65c1a29532ff878c506343ac
paper_authors.name: Di Zhang
paper_authors._id: 65c1a29532ff878c506343ad
paper_authors.name: Xiaodong Chen
paper_authors._id: 65c1a29532ff878c506343ae
paper_authors.name: Jing Liao
paper_title: Direct-a-Video: Customized Video Generation with User-Directed Camera
    Movement and Object Motion
paper_summary: Recent text-to-video diffusion models have achieved impressive progress. In
  practice, users often desire the ability to control object motion and camera
  movement independently for customized video creation. However, current methods
  lack the focus on separately controlling object motion and camera movement in a
  decoupled manner, which limits the controllability and flexibility of
  text-to-video models. In this paper, we introduce Direct-a-Video, a system that
  allows users to independently specify motions for one or multiple objects
  and/or camera movements, as if directing a video. We propose a simple yet
  effective strategy for the decoupled control of object motion and camera
  movement. Object motion is controlled through spatial cross-attention
  modulation using the model's inherent priors, requiring no additional
  optimization. For camera movement, we introduce new temporal cross-attention
  layers to interpret quantitative camera movement parameters. We further employ
  an augmentation-based approach to train these layers in a self-supervised
  manner on a small-scale dataset, eliminating the need for explicit motion
  annotation. Both components operate independently, allowing individual or
  combined control, and can generalize to open-domain scenarios. Extensive
  experiments demonstrate the superiority and effectiveness of our method.
  Project page: https://direct-a-video.github.io/.
paper_ai_keywords: diffusion models text-to-video independent control object motion camera movement spatial cross-attention temporal cross-attention quantitative camera movement parameters self-supervised learning augmentation-based approach open-domain scenarios