wav2pos: Sound Source Localization using Masked Autoencoders
<|reference_start|>wav2pos: Sound Source Localization using Masked Autoencoders: We present a novel approach to the 3D sound source localization task for distributed ad-hoc microphone arrays by formulating it as a set-to-set regression problem. By training a multi-modal masked autoencoder model that operates on audio recordings and microphone coordinates, we show that such a formulation allows for accurate localization of the sound source, by reconstructing coordinates masked in the input. Our approach is flexible in the sense that a single model can be used with an arbitrary number of microphones, even when a subset of audio recordings and microphone coordinates are missing. We test our method on simulated and real-world recordings of music and speech in indoor environments, and demonstrate competitive performance compared to both classical and other learning based localization methods.<|reference_end|>
@article{berg2024wav2pos:,
    title={wav2pos: Sound Source Localization using Masked Autoencoders},
    author={Axel Berg, Jens Gulin, Mark O'Connor, Chuteng Zhou, Karl {AA}str"om,
  Magnus Oskarsson},
    journal={arXiv preprint arXiv:2408.15771},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.15771},
    primaryClass={eess.AS cs.LG cs.SD}
}