Towards Ultra-Low-Power Neuromorphic Speech Enhancement with Spiking-FullSubNet
<|reference_start|>Towards Ultra-Low-Power Neuromorphic Speech Enhancement with Spiking-FullSubNet: Speech enhancement is critical for improving speech intelligibility and quality in various audio devices. In recent years, deep learning-based methods have significantly improved speech enhancement performance, but they often come with a high computational cost, which is prohibitive for a large number of edge devices, such as headsets and hearing aids. This work proposes an ultra-low-power speech enhancement system based on the brain-inspired spiking neural network (SNN) called Spiking-FullSubNet. Spiking-FullSubNet follows a full-band and sub-band fusioned approach to effectively capture both global and local spectral information. To enhance the efficiency of computationally expensive sub-band modeling, we introduce a frequency partitioning method inspired by the sensitivity profile of the human peripheral auditory system. Furthermore, we introduce a novel spiking neuron model that can dynamically control the input information integration and forgetting, enhancing the multi-scale temporal processing capability of SNN, which is critical for speech denoising. Experiments conducted on the recent Intel Neuromorphic Deep Noise Suppression (N-DNS) Challenge dataset show that the Spiking-FullSubNet surpasses state-of-the-art methods by large margins in terms of both speech quality and energy efficiency metrics. Notably, our system won the championship of the Intel N-DNS Challenge (Algorithmic Track), opening up a myriad of opportunities for ultra-low-power speech enhancement at the edge. Our source code and model checkpoints are publicly available at https://github.com/haoxiangsnr/spiking-fullsubnet.<|reference_end|>
@article{hao2024towards,
    title={Towards Ultra-Low-Power Neuromorphic Speech Enhancement with
  Spiking-FullSubNet},
    author={Xiang Hao, Chenxiang Ma, Qu Yang, Jibin Wu, Kay Chen Tan},
    journal={arXiv preprint arXiv:2410.04785},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.04785},
    primaryClass={eess.AS cs.SD}
}