Real-time speech recognition on mobile and embedded devices is an important application of neural networks. Acoustic modeling is the fundamental part of speech recognition and is usually implemented with long short-term memory (LSTM)-based recurrent neural networks (RNNs). However, the single thread execution of an LSTM RNN is extremely slow in most embedded devices because the algorithm needs to fetch a large number of parameters from the DRAM for computing each output sample. We explore a few acoustic modeling algorithms that can be executed very efficiently on embedded devices. These algorithms reduce the overhead of memory accesses using multi-timestep parallelization that computes multiple output samples at a time by reading the parameters only once from the DRAM. The algorithms considered are the quasi RNNs (QRNNs), Gated ConvNets, and diagonalized LSTMs. In addition, we explore neural networks that equip one-dimensional (1-D) convolution at each layer of these algorithms, and by which can obtain a very large performance increase in the QRNNs and Gated ConvNets. The experiments were conducted using two tasks, one is the connectionist temporal classification (CTC)-based end-to-end speech recognition on WSJ corpus and the other is the phoneme classification on TIMIT dataset. We not only significantly increase the execution speed but also obtain a much higher accuracy, compared to LSTM RNN-based modeling. Thus, this work can be applicable not only to embedded system-based implementations but also to server-based ones.
This paper discusses applications of variants of RNNs and Gated CNN to acoustic modeling in embedded speech recognition systems, and the main focus of the paper is computational (memory) efficiency when we deploy the system. The paper well describes the problem of the current LSTM, especially focusing on the recurrent connection matrix operations, which is a bottle neck in this scenario, and introduces variants of RNNs (e.g., QRNN). Also these variants may not yield enough performance compared with LSTM, but 1-D convolution and/or deep structure helps to avoid the degradation. One of the biggest issues of this paper is that they use CTC as an acoustic model, while still many real speech recognition applications and major open source (Kaldi) use hybrid HMM/DNN(TDNN, LSTM, CNN, etc.) systems. Therefore, the paper's claim on CTC is not along with the current application trends. (It may be changed near future, but still hybrid systems are dominant). For example, the WSJ WER performance listed in Table 3 is easily obtained by a simple feed-forward DNN in the hybrid system. The latest Lattice free MMI with TDNN can achieve better performance (~2.X% WER), and this decoding is quite fast compared with LSTM. The authors should consider this current situation of state-of-the-art speech recognition. Also, the techniques described in the paper are all based on existing techniques, and the paper lacks the technical novelty.
Other comments:
- in Abstract and the first part of Introduction: as I mentioned above, CTC based character-prediction modeling is not a major acoustic model.
- The paper needs some discussions about TDNN, which is a major acoustic modeling (fast and accurate) in Kaldi
- p.4 first line "and  represents element-wise multiplication": The element-wise multiplication operation was first appeared in Eq. (1), and it should be explained there.
- Section 3.2: I actually don't fully understand the claims of this experiment based on TIMIT, as it is phoneme recognition, and not directly related to the real application, which is the main target of this paper I think. My suggestion is to place these TIMIT based experiments as a preliminary experiment to investigate the variants of RNN or gated CNN before the WSJ experiments. (I did not say that Section 3.2 is useless. This analysis is actually valuable, and this suggested change about the position of this TIMIT experiment can avoid some confusion of the main target of this paper.)