better than a typical Mandarin Chinese speaker. To benchmark against humans we ran a test with
100 randomly selected utterances and had a group of 5 humans label all of them together. The group
of humans had an error rate of 4.0% as compared to the speech systems performance of 3.7%. We
also compared a single human transcriber to the speech system on 250 randomly selected utterances.
In this case the speech system performs much better: 9.7% for the human compared to 5.7% for the
speech model.
7 Deployment
Real-world applications usually require a speech system to transcribe in real time or with relatively
low latency. The system used in Section 6.1 is not well-designed for this task, for several reasons.
First, since the RNN has several bidirectional layers, transcribing the ﬁrst part of an utterance requires the entire utterance to be presented to the RNN. Second, since we use a wide beam when
decoding with a language model, beam search can be expensive, particularly in Mandarin where the
number of possible next characters is very large (around 6000). Third, as described in Section 3, we
normalize power across an entire utterance, which again requires the entire utterance to be available
in advance.
We solve the power normalization problem by using some statistics from our training set to perform
We show that an end-to-end deep learning approach can be used to recognize
either English or Mandarin Chinese speech--two vastly different languages.
Because it replaces entire pipelines of hand-engineered components with neural
networks, end-to-end learning allows us to handle a diverse variety of speech
including noisy environments, accents and different languages. Key to our
approach is our application of HPC techniques, resulting in a 7x speedup over
our previous system. Because of this efficiency, experiments that previously
took weeks now run in days. This enables us to iterate more quickly to identify
superior architectures and algorithms. As a result, in several cases, our
system is competitive with the transcription of human workers when benchmarked
on standard datasets. Finally, using a technique called Batch Dispatch with
GPUs in the data center, we show that our system can be inexpensively deployed
in an online setting, delivering low latency when serving users at scale.
Dario Amodei Rishita Anubhai Eric Battenberg Carl Case Jared Casper Bryan Catanzaro Jingdong Chen Mike Chrzanowski Adam Coates Greg Diamos Erich Elsen Jesse Engel Linxi Fan Christopher Fougner Tony Han Awni Hannun Billy Jun Patrick LeGresley Libby Lin Sharan Narang Andrew Ng Sherjil Ozair Ryan Prenger Jonathan Raiman Sanjeev Satheesh David Seetapun Shubho Sengupta Yi Wang Zhiqian Wang Chong Wang Bo Xiao Dani Yogatama Jun Zhan Zhenyao Zhu