Using Speech Foundational Models in Loss Functions for Hearing Aid Speech Enhancement
<|reference_start|>Using Speech Foundational Models in Loss Functions for Hearing Aid Speech Enhancement: Machine learning techniques are an active area of research for speech enhancement for hearing aids, with one particular focus on improving the intelligibility of a noisy speech signal. Recent work has shown that feature encodings from self-supervised speech representation models can effectively capture speech intelligibility. In this work, it is shown that the distance between self-supervised speech representations of clean and noisy speech correlates more strongly with human intelligibility ratings than other signal-based metrics. Experiments show that training a speech enhancement model using this distance as part of a loss function improves the performance over using an SNR-based loss function, demonstrated by an increase in HASPI, STOI, PESQ and SI-SNR scores. This method takes inference of a high parameter count model only at training time, meaning the speech enhancement model can remain smaller, as is required for hearing aids.<|reference_end|>
@article{sutherland2024using,
    title={Using Speech Foundational Models in Loss Functions for Hearing Aid
  Speech Enhancement},
    author={Robert Sutherland, George Close, Thomas Hain, Stefan Goetze, Jon
  Barker},
    journal={arXiv preprint arXiv:2407.13333},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.13333},
    primaryClass={cs.SD eess.AS}
}