A lightweight dual-stage framework for personalized speech enhancement based on DeepFilterNet2
<|reference_start|>A lightweight dual-stage framework for personalized speech enhancement based on DeepFilterNet2: Isolating the desired speaker's voice amidst multiplespeakers in a noisy acoustic context is a challenging task. Per-sonalized speech enhancement (PSE) endeavours to achievethis by leveraging prior knowledge of the speaker's voice.Recent research efforts have yielded promising PSE mod-els, albeit often accompanied by computationally intensivearchitectures, unsuitable for resource-constrained embeddeddevices. In this paper, we introduce a novel method to per-sonalize a lightweight dual-stage Speech Enhancement (SE)model and implement it within DeepFilterNet2, a SE modelrenowned for its state-of-the-art performance. We seek anoptimal integration of speaker information within the model,exploring different positions for the integration of the speakerembeddings within the dual-stage enhancement architec-ture. We also investigate a tailored training strategy whenadapting DeepFilterNet2 to a PSE task. We show that ourpersonalization method greatly improves the performancesof DeepFilterNet2 while preserving minimal computationaloverhead.<|reference_end|>
@article{serre2024a,
    title={A lightweight dual-stage framework for personalized speech enhancement
  based on DeepFilterNet2},
    author={Thomas Serre (S2A, IDS), Mathieu Fontaine (S2A, IDS), 'Eric Benhaim,
  Geoffroy Dutour, Slim Essid (S2A, IDS)},
    journal={ICASSP, Apr 2024, Seoul (Korea), South Korea},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.08022},
    primaryClass={cs.SD eess.AS}
}