title: CineMaster: A 3D-Aware and Controllable Framework for Cinematic
    Text-to-Video Generation
summary: In this work, we present CineMaster, a novel framework for 3D-aware and
  controllable text-to-video generation. Our goal is to empower users with
  comparable controllability as professional film directors: precise placement of
  objects within the scene, flexible manipulation of both objects and camera in
  3D space, and intuitive layout control over the rendered frames. To achieve
  this, CineMaster operates in two stages. In the first stage, we design an
  interactive workflow that allows users to intuitively construct 3D-aware
  conditional signals by positioning object bounding boxes and defining camera
  movements within the 3D space. In the second stage, these control
  signals--comprising rendered depth maps, camera trajectories and object class
  labels--serve as the guidance for a text-to-video diffusion model, ensuring to
  generate the user-intended video content. Furthermore, to overcome the scarcity
  of in-the-wild datasets with 3D object motion and camera pose annotations, we
  carefully establish an automated data annotation pipeline that extracts 3D
  bounding boxes and camera trajectories from large-scale video data. Extensive
  qualitative and quantitative experiments demonstrate that CineMaster
  significantly outperforms existing methods and implements prominent 3D-aware
  text-to-video generation. Project page: https://cinemaster-dev.github.io/.
paper_authors._id: 67ad5f25cad644864b436186
paper_authors.name: Qinghe Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-31T08:16:20.407Z
paper_authors._id: 646f3418a6a58aa29505fd30
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/646f3418a6a58aa29505fd30/1z13rnpb6rsUgQsYumWPg.png
paper_authors.fullname: QINGHE WANG
paper_authors.type: user
paper_authors.user: DecoderWQH666
paper_authors._id: 67ad5f25cad644864b436187
paper_authors.name: Yawen Luo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-14T15:30:49.636Z
paper_authors._id: 66743477ab975c859114d410
paper_authors.avatarUrl: /avatars/ac692cc336e383fb2cb53db6d1e3fe8c.svg
paper_authors.fullname: yawenluo
paper_authors.type: user
paper_authors.user: yawenluo
paper_authors._id: 67ad5f25cad644864b436188
paper_authors.name: Xiaoyu Shi
paper_authors._id: 67ad5f25cad644864b436189
paper_authors.name: Xu Jia
paper_authors._id: 67ad5f25cad644864b43618a
paper_authors.name: Huchuan Lu
paper_authors._id: 67ad5f25cad644864b43618b
paper_authors.name: Tianfan Xue
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-14T15:31:01.219Z
paper_authors._id: 67631f57abfbd60470d4b3c3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/5VrG0IZjYrDLiatOr4y06.png
paper_authors.fullname: Tianfan Xue
paper_authors.type: user
paper_authors.user: littlemouse9
paper_authors._id: 67ad5f25cad644864b43618c
paper_authors.name: Xintao Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-14T15:31:06.645Z
paper_authors._id: 60e272ca6c78a8c122b12127
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60e272ca6c78a8c122b12127/xldEGBzGrU-bX6IwAw0Ie.jpeg
paper_authors.fullname: Xintao Wang
paper_authors.type: user
paper_authors.user: Xintao
paper_authors._id: 67ad5f25cad644864b43618d
paper_authors.name: Pengfei Wan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-17T14:42:49.297Z
paper_authors._id: 662f93942510ef5735d7ad00
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/662f93942510ef5735d7ad00/ZIDIPm63sncIHFTT5b0uR.png
paper_authors.fullname: magicwpf
paper_authors.type: user
paper_authors.user: magicwpf
paper_authors._id: 67ad5f25cad644864b43618e
paper_authors.name: Di Zhang
paper_authors._id: 67ad5f25cad644864b43618f
paper_authors.name: Kun Gai
paper_title: CineMaster: A 3D-Aware and Controllable Framework for Cinematic
    Text-to-Video Generation
paper_summary: In this work, we present CineMaster, a novel framework for 3D-aware and
  controllable text-to-video generation. Our goal is to empower users with
  comparable controllability as professional film directors: precise placement of
  objects within the scene, flexible manipulation of both objects and camera in
  3D space, and intuitive layout control over the rendered frames. To achieve
  this, CineMaster operates in two stages. In the first stage, we design an
  interactive workflow that allows users to intuitively construct 3D-aware
  conditional signals by positioning object bounding boxes and defining camera
  movements within the 3D space. In the second stage, these control
  signals--comprising rendered depth maps, camera trajectories and object class
  labels--serve as the guidance for a text-to-video diffusion model, ensuring to
  generate the user-intended video content. Furthermore, to overcome the scarcity
  of in-the-wild datasets with 3D object motion and camera pose annotations, we
  carefully establish an automated data annotation pipeline that extracts 3D
  bounding boxes and camera trajectories from large-scale video data. Extensive
  qualitative and quantitative experiments demonstrate that CineMaster
  significantly outperforms existing methods and implements prominent 3D-aware
  text-to-video generation. Project page: https://cinemaster-dev.github.io/.
paper_ai_summary: CineMaster is a framework for 3D-aware and controllable text-to-video generation using user-defined control signals and a text-to-video diffusion model.
paper_ai_keywords: 3D-aware text-to-video generation interactive workflow bounding boxes camera movements conditional signals depth maps camera trajectories object class labels text-to-video diffusion model automated data annotation pipeline 3D bounding boxes