Flying with Photons: Rendering Novel Views of Propagating Light
<|reference_start|>Flying with Photons: Rendering Novel Views of Propagating Light: We present an imaging and neural rendering technique that seeks to synthesize videos of light propagating through a scene from novel, moving camera viewpoints. Our approach relies on a new ultrafast imaging setup to capture a first-of-its kind, multi-viewpoint video dataset with picosecond-level temporal resolution. Combined with this dataset, we introduce an efficient neural volume rendering framework based on the transient field. This field is defined as a mapping from a 3D point and 2D direction to a high-dimensional, discrete-time signal that represents time-varying radiance at ultrafast timescales. Rendering with transient fields naturally accounts for effects due to the finite speed of light, including viewpoint-dependent appearance changes caused by light propagation delays to the camera. We render a range of complex effects, including scattering, specular reflection, refraction, and diffraction. Additionally, we demonstrate removing viewpoint-dependent propagation delays using a time warping procedure, rendering of relativistic effects, and video synthesis of direct and global components of light transport.<|reference_end|>
@article{malik2024flying,
    title={Flying with Photons: Rendering Novel Views of Propagating Light},
    author={Anagh Malik, Noah Juravsky, Ryan Po, Gordon Wetzstein, Kiriakos N.
  Kutulakos, David B. Lindell},
    journal={arXiv preprint arXiv:2404.06493},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.06493},
    primaryClass={cs.CV eess.IV}
}