alignment. In principle, CTC could choose to emit all the characters of the transcription after some
15
ﬁxed delay and this can happen with unidirectional RNNs [54]. However, we found that CTC
produces an accurate alignment when trained with a bidirectional RNN.
Following the alignment is a segmentation step that splices the audio and the corresponding aligned
transcription whenever it encounters a long series of consecutive blank labels occurs, since this
usually denotes a stretch of silence. By tuning the number of consecutive blank s, we can tune the
length of the utterances generated. For the English speech data, we also require a space token to be
within the stretch of blank s in order to segment only on word boundaries. We tune the segmentation
to generate utterances that are on average 7 seconds long.
The ﬁnal step in the pipeline removes erroneous examples that arise from a failed alignment. We
crowd source the ground truth transcriptions for several thousand examples. The word level edit
distance between the ground truth and the aligned transcription is used to produce a good orbad
label. The threshold for the word level edit distance is chosen such that the resulting WER of the
good portion of the development set is less than 5%. We then train a linear classiﬁer to accurately
We show that an end-to-end deep learning approach can be used to recognize
either English or Mandarin Chinese speech--two vastly different languages.
Because it replaces entire pipelines of hand-engineered components with neural
networks, end-to-end learning allows us to handle a diverse variety of speech
including noisy environments, accents and different languages. Key to our
approach is our application of HPC techniques, resulting in a 7x speedup over
our previous system. Because of this efficiency, experiments that previously
took weeks now run in days. This enables us to iterate more quickly to identify
superior architectures and algorithms. As a result, in several cases, our
system is competitive with the transcription of human workers when benchmarked
on standard datasets. Finally, using a technique called Batch Dispatch with
GPUs in the data center, we show that our system can be inexpensively deployed
in an online setting, delivering low latency when serving users at scale.
Dario Amodei Rishita Anubhai Eric Battenberg Carl Case Jared Casper Bryan Catanzaro Jingdong Chen Mike Chrzanowski Adam Coates Greg Diamos Erich Elsen Jesse Engel Linxi Fan Christopher Fougner Tony Han Awni Hannun Billy Jun Patrick LeGresley Libby Lin Sharan Narang Andrew Ng Sherjil Ozair Ryan Prenger Jonathan Raiman Sanjeev Satheesh David Seetapun Shubho Sengupta Yi Wang Zhiqian Wang Chong Wang Bo Xiao Dani Yogatama Jun Zhan Zhenyao Zhu