title: EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video
    Guidance
summary: Recent approaches on 3D camera control in video diffusion models (VDMs) often
  create anchor videos to guide diffusion models as a structured prior by
  rendering from estimated point clouds following annotated camera trajectories.
  However, errors inherent in point cloud estimation often lead to inaccurate
  anchor videos. Moreover, the requirement for extensive camera trajectory
  annotations further increases resource demands. To address these limitations,
  we introduce EPiC, an efficient and precise camera control learning framework
  that automatically constructs high-quality anchor videos without expensive
  camera trajectory annotations. Concretely, we create highly precise anchor
  videos for training by masking source videos based on first-frame visibility.
  This approach ensures high alignment, eliminates the need for camera trajectory
  annotations, and thus can be readily applied to any in-the-wild video to
  generate image-to-video (I2V) training pairs. Furthermore, we introduce
  Anchor-ControlNet, a lightweight conditioning module that integrates anchor
  video guidance in visible regions to pretrained VDMs, with less than 1% of
  backbone model parameters. By combining the proposed anchor video data and
  ControlNet module, EPiC achieves efficient training with substantially fewer
  parameters, training steps, and less data, without requiring modifications to
  the diffusion model backbone typically needed to mitigate rendering
  misalignments. Although being trained on masking-based anchor videos, our
  method generalizes robustly to anchor videos made with point clouds during
  inference, enabling precise 3D-informed camera control. EPiC achieves SOTA
  performance on RealEstate10K and MiraData for I2V camera control task,
  demonstrating precise and robust camera control ability both quantitatively and
  qualitatively. Notably, EPiC also exhibits strong zero-shot generalization to
  video-to-video scenarios.
paper_authors._id: 6837d80bf42b2aacfc26c460
paper_authors.name: Zun Wang
paper_authors._id: 6837d80bf42b2aacfc26c461
paper_authors.name: Jaemin Cho
paper_authors._id: 6837d80bf42b2aacfc26c462
paper_authors.name: Jialu Li
paper_authors._id: 6837d80bf42b2aacfc26c463
paper_authors.name: Han Lin
paper_authors._id: 6837d80bf42b2aacfc26c464
paper_authors.name: Jaehong Yoon
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-29T07:42:07.590Z
paper_authors._id: 652066649004117947e46ed6
paper_authors.avatarUrl: /avatars/972c97df6f26d2c3d6ce71ec579984bb.svg
paper_authors.fullname: Jaehong Yoon
paper_authors.type: user
paper_authors.user: jaehong31
paper_authors._id: 6837d80bf42b2aacfc26c465
paper_authors.name: Yue Zhang
paper_authors._id: 6837d80bf42b2aacfc26c466
paper_authors.name: Mohit Bansal
paper_title: EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video
    Guidance
paper_summary: Recent approaches on 3D camera control in video diffusion models (VDMs) often
  create anchor videos to guide diffusion models as a structured prior by
  rendering from estimated point clouds following annotated camera trajectories.
  However, errors inherent in point cloud estimation often lead to inaccurate
  anchor videos. Moreover, the requirement for extensive camera trajectory
  annotations further increases resource demands. To address these limitations,
  we introduce EPiC, an efficient and precise camera control learning framework
  that automatically constructs high-quality anchor videos without expensive
  camera trajectory annotations. Concretely, we create highly precise anchor
  videos for training by masking source videos based on first-frame visibility.
  This approach ensures high alignment, eliminates the need for camera trajectory
  annotations, and thus can be readily applied to any in-the-wild video to
  generate image-to-video (I2V) training pairs. Furthermore, we introduce
  Anchor-ControlNet, a lightweight conditioning module that integrates anchor
  video guidance in visible regions to pretrained VDMs, with less than 1% of
  backbone model parameters. By combining the proposed anchor video data and
  ControlNet module, EPiC achieves efficient training with substantially fewer
  parameters, training steps, and less data, without requiring modifications to
  the diffusion model backbone typically needed to mitigate rendering
  misalignments. Although being trained on masking-based anchor videos, our
  method generalizes robustly to anchor videos made with point clouds during
  inference, enabling precise 3D-informed camera control. EPiC achieves SOTA
  performance on RealEstate10K and MiraData for I2V camera control task,
  demonstrating precise and robust camera control ability both quantitatively and
  qualitatively. Notably, EPiC also exhibits strong zero-shot generalization to
  video-to-video scenarios.
paper_ai_summary: EPiC is a framework for efficient 3D camera control in video diffusion models that constructs high-quality anchor videos through first-frame visibility masking, integrates them using a lightweight ControlNet module, and achieves state-of-the-art performance on I2V tasks with minimal resources.
paper_ai_keywords: anchor videos point cloud estimation camera trajectories diffusion models first-frame visibility EPiC ControlNet I2V training pairs rendering misalignments RealEstate10K MiraData