Acoustic to Word Recognition with Sequence to Sequence Models
Acoustic to Word recognition provides a straightforward solution to end to end speech recognition without needing external decoding, language model re scoring or lexicon. While character based models offer a natural solution to the of vocabulary problem, word models can be simpler to decode and may also be able to directly recognize semantically meaningful units. We present effective methods to train Sequence to Sequence models for direct word level recognition (and character level recognition) and show an absolute improvement of 4.4 5.0% in Word Error Rate on the Switchboard corpus compared to prior work. In addition to these promising results, word based models are more interpretable than character models, which have to be composed into words using a separate decoding step. We analyze the encoder hidden states and the attention behavior, and show that location aware attention naturally represents words as a single speech word vector, despite spanning multiple frames in the input. We finally show that the Acoustic to Word model also learns to segment speech into words with a mean standard deviation of 3 frames as compared with human annotated forced alignments for the Switchboard corpus.
2018 IEEE Spoken Language Technology Workshop (SLT)