Text Injection for Capitalization and Turn-Taking Prediction in Speech
  Models
Text injection for automatic speech recognition (ASR), wherein unpaired
text-only data is used to supplement paired audio-text data, has shown
promising improvements for word error rate. This study examines the use of text
injection for auxiliary tasks, which are the non-ASR tasks often performed by
an E2E model. In this work, we use joint end-to-end and internal language model
training (JEIT) as our text injection algorithm to train an ASR model which
performs two auxiliary tasks. The first is capitalization, which is a
de-normalization task. The second is turn-taking prediction, which attempts to
identify whether a user has completed their conversation turn in a digital
assistant interaction. We show results demonstrating that our text injection
method boosts capitalization performance for long-tail data, and improves
turn-taking detection recall.
64dc797c0fd0857facb54ecd
Shaan Bijwadia
64dc797c0fd0857facb54ece
Shuo-yiin Chang
64dc797c0fd0857facb54ecf
Weiran Wang
64dc797c0fd0857facb54ed0
Zhong Meng
64dc797c0fd0857facb54ed1
Hao Zhang
64dc797c0fd0857facb54ed2
Tara N. Sainath
Text Injection for Capitalization and Turn-Taking Prediction in Speech
  Models
Text injection for automatic speech recognition (ASR), wherein unpaired
text-only data is used to supplement paired audio-text data, has shown
promising improvements for word error rate. This study examines the use of text
injection for auxiliary tasks, which are the non-ASR tasks often performed by
an E2E model. In this work, we use joint end-to-end and internal language model
training (JEIT) as our text injection algorithm to train an ASR model which
performs two auxiliary tasks. The first is capitalization, which is a
de-normalization task. The second is turn-taking prediction, which attempts to
identify whether a user has completed their conversation turn in a digital
assistant interaction. We show results demonstrating that our text injection
method boosts capitalization performance for long-tail data, and improves
turn-taking detection recall.
Text injection using joint end-to-end and internal language model training enhances capitalization and turn-taking detection in automatic speech recognition systems.
joint end-to-end and internal language model training (JEIT) capitalization de-normalization turn-taking prediction automatic speech recognition (ASR)