Denoising LM: Pushing the Limits of Error Correction Models for Speech
  Recognition
Language models (LMs) have long been used to improve results of automatic
speech recognition (ASR) systems, but they are unaware of the errors that ASR
systems make. Error correction models are designed to fix ASR errors, however,
they showed little improvement over traditional LMs mainly due to the lack of
supervised training data. In this paper, we present Denoising LM (DLM), which
is a scaled error correction model trained with vast amounts of
synthetic data, significantly exceeding prior attempts meanwhile achieving new
state-of-the-art ASR performance. We use text-to-speech (TTS) systems to
synthesize audio, which is fed into an ASR system to produce noisy hypotheses,
which are then paired with the original texts to train the DLM. DLM has several
key ingredients: (i) up-scaled model and data; (ii) usage of
multi-speaker TTS systems; (iii) combination of multiple noise augmentation
strategies; and (iv) new decoding techniques. With a Transformer-CTC ASR, DLM
achieves 1.5% word error rate (WER) on test-clean and 3.3% WER on
test-other on Librispeech, which to our knowledge are the best
reported numbers in the setting where no external audio data are used and even
match self-supervised methods which use external audio data. Furthermore, a
single DLM is applicable to different ASRs, and greatly surpassing the
performance of conventional LM based beam-search rescoring. These results
indicate that properly investigated error correction models have the potential
to replace conventional LMs, holding the key to a new level of accuracy in ASR
systems.
6653eee1edd332b9443d2ada
Zijin Gu
admin_assigned
2024-05-27T09:50:52.499Z
6622d179ef00d8ae7a991bc1
/avatars/8dfcdc14605cc47664c804b0ba5fc444.svg
Zijin Gu
user
lelegu
6653eee1edd332b9443d2adb
Tatiana Likhomanenko
6653eee1edd332b9443d2adc
He Bai
claimed_verified
2024-05-28T07:19:14.964Z
5f024bd8ff69163f6f59e775
https://cdn-avatars.huggingface.co/v1/production/uploads/1652111948107-5f024bd8ff69163f6f59e775.png
Richard He Bai
user
richardbaihe
6653eee1edd332b9443d2add
Erik McDermott
6653eee1edd332b9443d2ade
Ronan Collobert
6653eee1edd332b9443d2adf
Navdeep Jaitly
admin_assigned
2024-05-27T09:51:31.399Z
631119df88942700629f4aab
/avatars/251c20a8fd48831a8c0cc0d198a42455.svg
Navdeep Jaitly
user
ndjaitly
Denoising LM: Pushing the Limits of Error Correction Models for Speech
  Recognition
Language models (LMs) have long been used to improve results of automatic
speech recognition (ASR) systems, but they are unaware of the errors that ASR
systems make. Error correction models are designed to fix ASR errors, however,
they showed little improvement over traditional LMs mainly due to the lack of
supervised training data. In this paper, we present Denoising LM (DLM), which
is a scaled error correction model trained with vast amounts of
synthetic data, significantly exceeding prior attempts meanwhile achieving new
state-of-the-art ASR performance. We use text-to-speech (TTS) systems to
synthesize audio, which is fed into an ASR system to produce noisy hypotheses,
which are then paired with the original texts to train the DLM. DLM has several
key ingredients: (i) up-scaled model and data; (ii) usage of
multi-speaker TTS systems; (iii) combination of multiple noise augmentation
strategies; and (iv) new decoding techniques. With a Transformer-CTC ASR, DLM
achieves 1.5% word error rate (WER) on test-clean and 3.3% WER on
test-other on Librispeech, which to our knowledge are the best
reported numbers in the setting where no external audio data are used and even
match self-supervised methods which use external audio data. Furthermore, a
single DLM is applicable to different ASRs, and greatly surpassing the
performance of conventional LM based beam-search rescoring. These results
indicate that properly investigated error correction models have the potential
to replace conventional LMs, holding the key to a new level of accuracy in ASR
systems.
Denoising language models (DLMs) trained with synthetic data achieve state-of-the-art ASR performance on Librispeech without external audio data, surpassing conventional language models and some self-supervised methods.
Denoising LM scaled error correction model text-to-speech systems ASR errors synthetic data multi-speaker TTS systems noise augmentation strategies Transformer-CTC ASR word error rate (WER) test-clean test-other Librispeech beam-search rescoring