title: Failing Forward: Improving Generative Error Correction for ASR with
    Synthetic Data and Retrieval Augmentation
summary: Generative Error Correction (GEC) has emerged as a powerful post-processing
  method to enhance the performance of Automatic Speech Recognition (ASR)
  systems. However, we show that GEC models struggle to generalize beyond the
  specific types of errors encountered during training, limiting their ability to
  correct new, unseen errors at test time, particularly in out-of-domain (OOD)
  scenarios. This phenomenon amplifies with named entities (NEs), where, in
  addition to insufficient contextual information or knowledge about the NEs,
  novel NEs keep emerging. To address these issues, we propose DARAG (Data- and
  Retrieval-Augmented Generative Error Correction), a novel approach designed to
  improve GEC for ASR in in-domain (ID) and OOD scenarios. We augment the GEC
  training dataset with synthetic data generated by prompting LLMs and
  text-to-speech models, thereby simulating additional errors from which the
  model can learn. For OOD scenarios, we simulate test-time errors from new
  domains similarly and in an unsupervised fashion. Additionally, to better
  handle named entities, we introduce retrieval-augmented correction by
  augmenting the input with entities retrieved from a database. Our approach is
  simple, scalable, and both domain- and language-agnostic. We experiment on
  multiple datasets and settings, showing that DARAG outperforms all our
  baselines, achieving 8\% -- 30\% relative WER improvements in ID and 10\% --
  33\% improvements in OOD settings.
paper_authors._id: 6711bfbd035ddf7e14b22417
paper_authors.name: Sreyan Ghosh
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-28T15:36:39.302Z
paper_authors._id: 62c9664eb34e600d7eaa4beb
paper_authors.avatarUrl: /avatars/ca23ecdec2d31c99ecce97d9b180ae0c.svg
paper_authors.fullname: Ghosh
paper_authors.type: user
paper_authors.user: Sreyan88
paper_authors._id: 6711bfbd035ddf7e14b22418
paper_authors.name: Mohammad Sadegh Rasooli
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:58:14.910Z
paper_authors._id: 658dd50b315340de5fcb1321
paper_authors.avatarUrl: /avatars/72b0030ff3f5f47002e0ff4e2d665b46.svg
paper_authors.fullname: Mohammad Sadegh Rasooli
paper_authors.type: user
paper_authors.user: rasoolims
paper_authors._id: 6711bfbd035ddf7e14b22419
paper_authors.name: Michael Levit
paper_authors._id: 6711bfbd035ddf7e14b2241a
paper_authors.name: Peidong Wang
paper_authors._id: 6711bfbd035ddf7e14b2241b
paper_authors.name: Jian Xue
paper_authors._id: 6711bfbd035ddf7e14b2241c
paper_authors.name: Dinesh Manocha
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T12:59:11.285Z
paper_authors._id: 6537a569568d8be8fa096b8c
paper_authors.avatarUrl: /avatars/bfda5cb252d8b5bc3ad737d99c0d7f49.svg
paper_authors.fullname: Dinesh Manocha
paper_authors.type: user
paper_authors.user: manocha
paper_authors._id: 6711bfbd035ddf7e14b2241d
paper_authors.name: Jinyu Li
paper_title: Failing Forward: Improving Generative Error Correction for ASR with
    Synthetic Data and Retrieval Augmentation
paper_summary: Generative Error Correction (GEC) has emerged as a powerful post-processing
  method to enhance the performance of Automatic Speech Recognition (ASR)
  systems. However, we show that GEC models struggle to generalize beyond the
  specific types of errors encountered during training, limiting their ability to
  correct new, unseen errors at test time, particularly in out-of-domain (OOD)
  scenarios. This phenomenon amplifies with named entities (NEs), where, in
  addition to insufficient contextual information or knowledge about the NEs,
  novel NEs keep emerging. To address these issues, we propose DARAG (Data- and
  Retrieval-Augmented Generative Error Correction), a novel approach designed to
  improve GEC for ASR in in-domain (ID) and OOD scenarios. We augment the GEC
  training dataset with synthetic data generated by prompting LLMs and
  text-to-speech models, thereby simulating additional errors from which the
  model can learn. For OOD scenarios, we simulate test-time errors from new
  domains similarly and in an unsupervised fashion. Additionally, to better
  handle named entities, we introduce retrieval-augmented correction by
  augmenting the input with entities retrieved from a database. Our approach is
  simple, scalable, and both domain- and language-agnostic. We experiment on
  multiple datasets and settings, showing that DARAG outperforms all our
  baselines, achieving 8\% -- 30\% relative WER improvements in ID and 10\% --
  33\% improvements in OOD settings.
paper_ai_summary: DARAG enhances generative error correction for ASR by using synthetic data and retrieval-augmented methods to improve performance in both in-domain and out-of-domain scenarios.
paper_ai_keywords: Generative Error Correction (GEC) Automatic Speech Recognition (ASR) Named Entities (NEs) out-of-domain (OOD) in-domain (ID) LLMs text-to-speech models retrieval-augmented correction relative Word Error Rate (WER) improvements