Manifold Sampling for Differentiable Uncertainty in Radiance Fields
<|reference_start|>Manifold Sampling for Differentiable Uncertainty in Radiance Fields: Radiance fields are powerful and, hence, popular models for representing the appearance of complex scenes. Yet, constructing them based on image observations gives rise to ambiguities and uncertainties. We propose a versatile approach for learning Gaussian radiance fields with explicit and fine-grained uncertainty estimates that impose only little additional cost compared to uncertainty-agnostic training. Our key observation is that uncertainties can be modeled as a low-dimensional manifold in the space of radiance field parameters that is highly amenable to Monte Carlo sampling. Importantly, our uncertainties are differentiable and, thus, allow for gradient-based optimization of subsequent captures that optimally reduce ambiguities. We demonstrate state-of-the-art performance on next-best-view planning tasks, including high-dimensional illumination planning for optimal radiance field relighting quality.<|reference_end|>
@article{lyu2024manifold,
    title={Manifold Sampling for Differentiable Uncertainty in Radiance Fields},
    author={Linjie Lyu, Ayush Tewari, Marc Habermann, Shunsuke Saito, Michael
  Zollh"ofer, Thomas Leimk"uhler and Christian Theobalt},
    journal={arXiv preprint arXiv:2409.12661},
    year={2024},
    doi={10.1145/3680528.3687655},
    archivePrefix={arXiv},
    eprint={2409.12661},
    primaryClass={cs.CV cs.GR}
}