title: NormalCrafter: Learning Temporally Consistent Normals from Video
    Diffusion Priors
summary: Surface normal estimation serves as a cornerstone for a spectrum of computer
  vision applications. While numerous efforts have been devoted to static image
  scenarios, ensuring temporal coherence in video-based normal estimation remains
  a formidable challenge. Instead of merely augmenting existing methods with
  temporal components, we present NormalCrafter to leverage the inherent temporal
  priors of video diffusion models. To secure high-fidelity normal estimation
  across sequences, we propose Semantic Feature Regularization (SFR), which
  aligns diffusion features with semantic cues, encouraging the model to
  concentrate on the intrinsic semantics of the scene. Moreover, we introduce a
  two-stage training protocol that leverages both latent and pixel space learning
  to preserve spatial accuracy while maintaining long temporal context. Extensive
  evaluations demonstrate the efficacy of our method, showcasing a superior
  performance in generating temporally consistent normal sequences with intricate
  details from diverse videos.
paper_authors._id: 67ff1cc4372d6790b1b7da90
paper_authors.name: Yanrui Bin
paper_authors._id: 67ff1cc4372d6790b1b7da91
paper_authors.name: Wenbo Hu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-16T11:57:59.932Z
paper_authors._id: 657a7458afbb0117ba15c59f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/657a7458afbb0117ba15c59f/8_iwTS1UG_mKnfylFbLsY.jpeg
paper_authors.fullname: Wenbo Hu
paper_authors.type: user
paper_authors.user: wbhu-tc
paper_authors._id: 67ff1cc4372d6790b1b7da92
paper_authors.name: Haoyuan Wang
paper_authors._id: 67ff1cc4372d6790b1b7da93
paper_authors.name: Xinya Chen
paper_authors._id: 67ff1cc4372d6790b1b7da94
paper_authors.name: Bing Wang
paper_title: NormalCrafter: Learning Temporally Consistent Normals from Video
    Diffusion Priors
paper_summary: Surface normal estimation serves as a cornerstone for a spectrum of computer
  vision applications. While numerous efforts have been devoted to static image
  scenarios, ensuring temporal coherence in video-based normal estimation remains
  a formidable challenge. Instead of merely augmenting existing methods with
  temporal components, we present NormalCrafter to leverage the inherent temporal
  priors of video diffusion models. To secure high-fidelity normal estimation
  across sequences, we propose Semantic Feature Regularization (SFR), which
  aligns diffusion features with semantic cues, encouraging the model to
  concentrate on the intrinsic semantics of the scene. Moreover, we introduce a
  two-stage training protocol that leverages both latent and pixel space learning
  to preserve spatial accuracy while maintaining long temporal context. Extensive
  evaluations demonstrate the efficacy of our method, showcasing a superior
  performance in generating temporally consistent normal sequences with intricate
  details from diverse videos.
paper_ai_summary: A novel method, NormalCrafter, enhances temporal coherence in video-based surface normal estimation by utilizing video diffusion models with Semantic Feature Regularization and two-stage training in latent and pixel spaces.
paper_ai_keywords: video diffusion models Semantic Feature Regularization two-stage training latent space pixel space surface normal estimation temporal coherence