FoVNet: Configurable Field-of-View Speech Enhancement with Low Computation and Distortion for Smart Glasses
<|reference_start|>FoVNet: Configurable Field-of-View Speech Enhancement with Low Computation and Distortion for Smart Glasses: This paper presents a novel multi-channel speech enhancement approach, FoVNet, that enables highly efficient speech enhancement within a configurable field of view (FoV) of a smart-glasses user without needing specific target-talker(s) directions. It advances over prior works by enhancing all speakers within any given FoV, with a hybrid signal processing and deep learning approach designed with high computational efficiency. The neural network component is designed with ultra-low computation (about 50 MMACS). A multi-channel Wiener filter and a post-processing module are further used to improve perceptual quality. We evaluate our algorithm with a microphone array on smart glasses, providing a configurable, efficient solution for augmented hearing on energy-constrained devices. FoVNet excels in both computational efficiency and speech quality across multiple scenarios, making it a promising solution for smart glasses applications.<|reference_end|>
@article{xu2024fovnet:,
    title={FoVNet: Configurable Field-of-View Speech Enhancement with Low
  Computation and Distortion for Smart Glasses},
    author={Zhongweiyang Xu, Ali Aroudi, Ke Tan, Ashutosh Pandey, Jung-Suk Lee,
  Buye Xu, Francesco Nesta},
    journal={arXiv preprint arXiv:2408.06468},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.06468},
    primaryClass={cs.SD cs.MM eess.AS eess.SP}
}