title: Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls
    for Video Generation
summary: Camera and human motion controls have been extensively studied for video
  generation, but existing approaches typically address them separately,
  suffering from limited data with high-quality annotations for both aspects. To
  overcome this, we present Uni3C, a unified 3D-enhanced framework for precise
  control of both camera and human motion in video generation. Uni3C includes two
  key contributions. First, we propose a plug-and-play control module trained
  with a frozen video generative backbone, PCDController, which utilizes
  unprojected point clouds from monocular depth to achieve accurate camera
  control. By leveraging the strong 3D priors of point clouds and the powerful
  capacities of video foundational models, PCDController shows impressive
  generalization, performing well regardless of whether the inference backbone is
  frozen or fine-tuned. This flexibility enables different modules of Uni3C to be
  trained in specific domains, i.e., either camera control or human motion
  control, reducing the dependency on jointly annotated data. Second, we propose
  a jointly aligned 3D world guidance for the inference phase that seamlessly
  integrates both scenic point clouds and SMPL-X characters to unify the control
  signals for camera and human motion, respectively. Extensive experiments
  confirm that PCDController enjoys strong robustness in driving camera motion
  for fine-tuned backbones of video generation. Uni3C substantially outperforms
  competitors in both camera controllability and human motion quality.
  Additionally, we collect tailored validation sets featuring challenging camera
  movements and human actions to validate the effectiveness of our method.
paper_authors._id: 68075a2bdadb28dddc13dfc6
paper_authors.name: Chenjie Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-22T11:03:42.731Z
paper_authors._id: 64892bc82eafb7b91182bec5
paper_authors.avatarUrl: /avatars/c9eed91e60f3ea227b35e111d3fc4200.svg
paper_authors.fullname: chenjie cao
paper_authors.type: user
paper_authors.user: ewrfcas
paper_authors._id: 68075a2bdadb28dddc13dfc7
paper_authors.name: Jingkai Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-22T11:03:50.199Z
paper_authors._id: 6434caa64b34368fdb07da48
paper_authors.avatarUrl: /avatars/8da3d3d1e274ff4ad409234678b1b952.svg
paper_authors.fullname: Jingkai Zhou
paper_authors.type: user
paper_authors.user: theFoxofSky
paper_authors._id: 68075a2bdadb28dddc13dfc8
paper_authors.name: Shikai Li
paper_authors._id: 68075a2bdadb28dddc13dfc9
paper_authors.name: Jingyun Liang
paper_authors._id: 68075a2bdadb28dddc13dfca
paper_authors.name: Chaohui Yu
paper_authors._id: 68075a2bdadb28dddc13dfcb
paper_authors.name: Fan Wang
paper_authors._id: 68075a2bdadb28dddc13dfcc
paper_authors.name: Xiangyang Xue
paper_authors._id: 68075a2bdadb28dddc13dfcd
paper_authors.name: Yanwei Fu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-22T11:04:23.211Z
paper_authors._id: 6409fcc8f3dabf93824c84c6
paper_authors.avatarUrl: /avatars/dd8fd579630e50ba3058c3829604478e.svg
paper_authors.fullname: YANWEI
paper_authors.type: user
paper_authors.user: yanweifuture
paper_title: Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls
    for Video Generation
paper_summary: Camera and human motion controls have been extensively studied for video
  generation, but existing approaches typically address them separately,
  suffering from limited data with high-quality annotations for both aspects. To
  overcome this, we present Uni3C, a unified 3D-enhanced framework for precise
  control of both camera and human motion in video generation. Uni3C includes two
  key contributions. First, we propose a plug-and-play control module trained
  with a frozen video generative backbone, PCDController, which utilizes
  unprojected point clouds from monocular depth to achieve accurate camera
  control. By leveraging the strong 3D priors of point clouds and the powerful
  capacities of video foundational models, PCDController shows impressive
  generalization, performing well regardless of whether the inference backbone is
  frozen or fine-tuned. This flexibility enables different modules of Uni3C to be
  trained in specific domains, i.e., either camera control or human motion
  control, reducing the dependency on jointly annotated data. Second, we propose
  a jointly aligned 3D world guidance for the inference phase that seamlessly
  integrates both scenic point clouds and SMPL-X characters to unify the control
  signals for camera and human motion, respectively. Extensive experiments
  confirm that PCDController enjoys strong robustness in driving camera motion
  for fine-tuned backbones of video generation. Uni3C substantially outperforms
  competitors in both camera controllability and human motion quality.
  Additionally, we collect tailored validation sets featuring challenging camera
  movements and human actions to validate the effectiveness of our method.
paper_ai_summary: Uni3C, a unified 3D-enhanced framework, improves camera and human motion control in video generation by leveraging unprojected point clouds and SMPL-X characters for robust and precise control.
paper_ai_keywords: Uni3C PCDController monocular depth point clouds video generative backbone 3D priors SMPL-X 3D world guidance camera control human motion control