Deep Learning, Artificial Intelligence, Asr, Transformers, Speech Recognition.
intermediate ReLU layer size=2048. Decoder input word embedding dim=512. 3 1-D conv. layers each with kernel size=3, no max pooling is used for the decoder side 1-D convolution. 10 decoder Transformer blocks each with encoder-side multihead attention. This canonical model has about 223M parameters,