CineMaster: A 3D-Aware and Controllable Framework for Cinematic
  Text-to-Video Generation
In this work, we present CineMaster, a novel framework for 3D-aware and
controllable text-to-video generation. Our goal is to empower users with
comparable controllability as professional film directors: precise placement of
objects within the scene, flexible manipulation of both objects and camera in
3D space, and intuitive layout control over the rendered frames. To achieve
this, CineMaster operates in two stages. In the first stage, we design an
interactive workflow that allows users to intuitively construct 3D-aware
conditional signals by positioning object bounding boxes and defining camera
movements within the 3D space. In the second stage, these control
signals--comprising rendered depth maps, camera trajectories and object class
labels--serve as the guidance for a text-to-video diffusion model, ensuring to
generate the user-intended video content. Furthermore, to overcome the scarcity
of in-the-wild datasets with 3D object motion and camera pose annotations, we
carefully establish an automated data annotation pipeline that extracts 3D
bounding boxes and camera trajectories from large-scale video data. Extensive
qualitative and quantitative experiments demonstrate that CineMaster
significantly outperforms existing methods and implements prominent 3D-aware
text-to-video generation. Project page: https://cinemaster-dev.github.io/.
67ad5f25cad644864b436186
Qinghe Wang
claimed_verified
2025-03-31T08:16:20.407Z
646f3418a6a58aa29505fd30
https://cdn-avatars.huggingface.co/v1/production/uploads/646f3418a6a58aa29505fd30/1z13rnpb6rsUgQsYumWPg.png
QINGHE WANG
user
DecoderWQH666
67ad5f25cad644864b436187
Yawen Luo
admin_assigned
2025-02-14T15:30:49.636Z
66743477ab975c859114d410
/avatars/ac692cc336e383fb2cb53db6d1e3fe8c.svg
yawenluo
user
yawenluo
67ad5f25cad644864b436188
Xiaoyu Shi
67ad5f25cad644864b436189
Xu Jia
67ad5f25cad644864b43618a
Huchuan Lu
67ad5f25cad644864b43618b
Tianfan Xue
admin_assigned
2025-02-14T15:31:01.219Z
67631f57abfbd60470d4b3c3
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/5VrG0IZjYrDLiatOr4y06.png
Tianfan Xue
user
littlemouse9
67ad5f25cad644864b43618c
Xintao Wang
admin_assigned
2025-02-14T15:31:06.645Z
60e272ca6c78a8c122b12127
https://cdn-avatars.huggingface.co/v1/production/uploads/60e272ca6c78a8c122b12127/xldEGBzGrU-bX6IwAw0Ie.jpeg
Xintao Wang
user
Xintao
67ad5f25cad644864b43618d
Pengfei Wan
claimed_verified
2025-03-17T14:42:49.297Z
662f93942510ef5735d7ad00
https://cdn-avatars.huggingface.co/v1/production/uploads/662f93942510ef5735d7ad00/ZIDIPm63sncIHFTT5b0uR.png
magicwpf
user
magicwpf
67ad5f25cad644864b43618e
Di Zhang
67ad5f25cad644864b43618f
Kun Gai
CineMaster: A 3D-Aware and Controllable Framework for Cinematic
  Text-to-Video Generation
In this work, we present CineMaster, a novel framework for 3D-aware and
controllable text-to-video generation. Our goal is to empower users with
comparable controllability as professional film directors: precise placement of
objects within the scene, flexible manipulation of both objects and camera in
3D space, and intuitive layout control over the rendered frames. To achieve
this, CineMaster operates in two stages. In the first stage, we design an
interactive workflow that allows users to intuitively construct 3D-aware
conditional signals by positioning object bounding boxes and defining camera
movements within the 3D space. In the second stage, these control
signals--comprising rendered depth maps, camera trajectories and object class
labels--serve as the guidance for a text-to-video diffusion model, ensuring to
generate the user-intended video content. Furthermore, to overcome the scarcity
of in-the-wild datasets with 3D object motion and camera pose annotations, we
carefully establish an automated data annotation pipeline that extracts 3D
bounding boxes and camera trajectories from large-scale video data. Extensive
qualitative and quantitative experiments demonstrate that CineMaster
significantly outperforms existing methods and implements prominent 3D-aware
text-to-video generation. Project page: https://cinemaster-dev.github.io/.
CineMaster is a framework for 3D-aware and controllable text-to-video generation using user-defined control signals and a text-to-video diffusion model.
3D-aware text-to-video generation interactive workflow bounding boxes camera movements conditional signals depth maps camera trajectories object class labels text-to-video diffusion model automated data annotation pipeline 3D bounding boxes