Inverse Rendering of Glossy Objects via the Neural Plenoptic Function and Radiance Fields
<|reference_start|>Inverse Rendering of Glossy Objects via the Neural Plenoptic Function and Radiance Fields: Inverse rendering aims at recovering both geometry and materials of objects. It provides a more compatible reconstruction for conventional rendering engines, compared with the neural radiance fields (NeRFs). On the other hand, existing NeRF-based inverse rendering methods cannot handle glossy objects with local light interactions well, as they typically oversimplify the illumination as a 2D environmental map, which assumes infinite lights only. Observing the superiority of NeRFs in recovering radiance fields, we propose a novel 5D Neural Plenoptic Function (NeP) based on NeRFs and ray tracing, such that more accurate lighting-object interactions can be formulated via the rendering equation. We also design a material-aware cone sampling strategy to efficiently integrate lights inside the BRDF lobes with the help of pre-filtered radiance fields. Our method has two stages: the geometry of the target object and the pre-filtered environmental radiance fields are reconstructed in the first stage, and materials of the target object are estimated in the second stage with the proposed NeP and material-aware cone sampling strategy. Extensive experiments on the proposed real-world and synthetic datasets demonstrate that our method can reconstruct high-fidelity geometry/materials of challenging glossy objects with complex lighting interactions from nearby objects. Project webpage: https://whyy.site/paper/nep<|reference_end|>
@article{wang2024inverse,
    title={Inverse Rendering of Glossy Objects via the Neural Plenoptic Function
  and Radiance Fields},
    author={Haoyuan Wang, Wenbo Hu, Lei Zhu, Rynson W. H. Lau},
    journal={arXiv preprint arXiv:2403.16224},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.16224},
    primaryClass={cs.CV}
}