SOAF: Scene Occlusion-aware Neural Acoustic Field
<|reference_start|>SOAF: Scene Occlusion-aware Neural Acoustic Field: This paper tackles the problem of novel view audio-visual synthesis along an arbitrary trajectory in an indoor scene, given the audio-video recordings from other known trajectories of the scene. Existing methods often overlook the effect of room geometry, particularly wall occlusion to sound propagation, making them less accurate in multi-room environments. In this work, we propose a new approach called Scene Occlusion-aware Acoustic Field (SOAF) for accurate sound generation. Our approach derives a prior for sound energy field using distance-aware parametric sound-propagation modelling and then transforms it based on scene transmittance learned from the input video. We extract features from the local acoustic field centred around the receiver using a Fibonacci Sphere to generate binaural audio for novel views with a direction-aware attention mechanism. Extensive experiments on the real dataset RWAVS and the synthetic dataset SoundSpaces demonstrate that our method outperforms previous state-of-the-art techniques in audio generation. Project page: https://github.com/huiyu-gao/SOAF/.<|reference_end|>
@article{gao2024soaf:,
    title={SOAF: Scene Occlusion-aware Neural Acoustic Field},
    author={Huiyu Gao, Jiahao Ma, David Ahmedt-Aristizabal, Chuong Nguyen,
  Miaomiao Liu},
    journal={arXiv preprint arXiv:2407.02264},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.02264},
    primaryClass={cs.CV cs.SD eess.AS}
}