title: Mirror-NeRF: Learning Neural Radiance Fields for Mirrors with
    Whitted-Style Ray Tracing
summary: Recently, Neural Radiance Fields (NeRF) has exhibited significant success in
  novel view synthesis, surface reconstruction, etc. However, since no physical
  reflection is considered in its rendering pipeline, NeRF mistakes the
  reflection in the mirror as a separate virtual scene, leading to the inaccurate
  reconstruction of the mirror and multi-view inconsistent reflections in the
  mirror. In this paper, we present a novel neural rendering framework, named
  Mirror-NeRF, which is able to learn accurate geometry and reflection of the
  mirror and support various scene manipulation applications with mirrors, such
  as adding new objects or mirrors into the scene and synthesizing the
  reflections of these new objects in mirrors, controlling mirror roughness, etc.
  To achieve this goal, we propose a unified radiance field by introducing the
  reflection probability and tracing rays following the light transport model of
  Whitted Ray Tracing, and also develop several techniques to facilitate the
  learning process. Experiments and comparisons on both synthetic and real
  datasets demonstrate the superiority of our method. The code and supplementary
  material are available on the project webpage:
  https://zju3dv.github.io/Mirror-NeRF/.
paper_authors._id: 64d1db4b4e3372f9ebdf1273
paper_authors.name: Junyi Zeng
paper_authors._id: 64d1db4b4e3372f9ebdf1274
paper_authors.name: Chong Bao
paper_authors._id: 64d1db4b4e3372f9ebdf1275
paper_authors.name: Rui Chen
paper_authors._id: 64d1db4b4e3372f9ebdf1276
paper_authors.name: Zilong Dong
paper_authors._id: 64d1db4b4e3372f9ebdf1277
paper_authors.name: Guofeng Zhang
paper_authors._id: 64d1db4b4e3372f9ebdf1278
paper_authors.name: Hujun Bao
paper_authors._id: 64d1db4b4e3372f9ebdf1279
paper_authors.name: Zhaopeng Cui
paper_title: Mirror-NeRF: Learning Neural Radiance Fields for Mirrors with
    Whitted-Style Ray Tracing
paper_summary: Recently, Neural Radiance Fields (NeRF) has exhibited significant success in
  novel view synthesis, surface reconstruction, etc. However, since no physical
  reflection is considered in its rendering pipeline, NeRF mistakes the
  reflection in the mirror as a separate virtual scene, leading to the inaccurate
  reconstruction of the mirror and multi-view inconsistent reflections in the
  mirror. In this paper, we present a novel neural rendering framework, named
  Mirror-NeRF, which is able to learn accurate geometry and reflection of the
  mirror and support various scene manipulation applications with mirrors, such
  as adding new objects or mirrors into the scene and synthesizing the
  reflections of these new objects in mirrors, controlling mirror roughness, etc.
  To achieve this goal, we propose a unified radiance field by introducing the
  reflection probability and tracing rays following the light transport model of
  Whitted Ray Tracing, and also develop several techniques to facilitate the
  learning process. Experiments and comparisons on both synthetic and real
  datasets demonstrate the superiority of our method. The code and supplementary
  material are available on the project webpage:
  https://zju3dv.github.io/Mirror-NeRF/.
paper_ai_summary: Mirror-NeRF improves novel view synthesis and surface reconstruction by accurately modeling mirror reflections and geometry, using Whitted Ray Tracing and unified radiance fields.
paper_ai_keywords: Neural Radiance Fields NeRF mirror reflections novel view synthesis surface reconstruction unified radiance fields reflection probability Whitted Ray Tracing