Mirror-NeRF: Learning Neural Radiance Fields for Mirrors with
  Whitted-Style Ray Tracing
Recently, Neural Radiance Fields (NeRF) has exhibited significant success in
novel view synthesis, surface reconstruction, etc. However, since no physical
reflection is considered in its rendering pipeline, NeRF mistakes the
reflection in the mirror as a separate virtual scene, leading to the inaccurate
reconstruction of the mirror and multi-view inconsistent reflections in the
mirror. In this paper, we present a novel neural rendering framework, named
Mirror-NeRF, which is able to learn accurate geometry and reflection of the
mirror and support various scene manipulation applications with mirrors, such
as adding new objects or mirrors into the scene and synthesizing the
reflections of these new objects in mirrors, controlling mirror roughness, etc.
To achieve this goal, we propose a unified radiance field by introducing the
reflection probability and tracing rays following the light transport model of
Whitted Ray Tracing, and also develop several techniques to facilitate the
learning process. Experiments and comparisons on both synthetic and real
datasets demonstrate the superiority of our method. The code and supplementary
material are available on the project webpage:
https://zju3dv.github.io/Mirror-NeRF/.
64d1db4b4e3372f9ebdf1273
Junyi Zeng
64d1db4b4e3372f9ebdf1274
Chong Bao
64d1db4b4e3372f9ebdf1275
Rui Chen
64d1db4b4e3372f9ebdf1276
Zilong Dong
64d1db4b4e3372f9ebdf1277
Guofeng Zhang
64d1db4b4e3372f9ebdf1278
Hujun Bao
64d1db4b4e3372f9ebdf1279
Zhaopeng Cui
Mirror-NeRF: Learning Neural Radiance Fields for Mirrors with
  Whitted-Style Ray Tracing
Recently, Neural Radiance Fields (NeRF) has exhibited significant success in
novel view synthesis, surface reconstruction, etc. However, since no physical
reflection is considered in its rendering pipeline, NeRF mistakes the
reflection in the mirror as a separate virtual scene, leading to the inaccurate
reconstruction of the mirror and multi-view inconsistent reflections in the
mirror. In this paper, we present a novel neural rendering framework, named
Mirror-NeRF, which is able to learn accurate geometry and reflection of the
mirror and support various scene manipulation applications with mirrors, such
as adding new objects or mirrors into the scene and synthesizing the
reflections of these new objects in mirrors, controlling mirror roughness, etc.
To achieve this goal, we propose a unified radiance field by introducing the
reflection probability and tracing rays following the light transport model of
Whitted Ray Tracing, and also develop several techniques to facilitate the
learning process. Experiments and comparisons on both synthetic and real
datasets demonstrate the superiority of our method. The code and supplementary
material are available on the project webpage:
https://zju3dv.github.io/Mirror-NeRF/.
Mirror-NeRF improves novel view synthesis and surface reconstruction by accurately modeling mirror reflections and geometry, using Whitted Ray Tracing and unified radiance fields.
Neural Radiance Fields NeRF mirror reflections novel view synthesis surface reconstruction unified radiance fields reflection probability Whitted Ray Tracing