Deep Learning, Artificial Intelligence, Asr, Transformers, Speech Recognition.
language model over the bag of discovered acoustic units as it goes deeper in the encoder. For the encoder, 2-D convolutional blocks are used, with Layer Norms and ReLU after each convolutional layer. Each convolutional block contains K convolutional layers followed by a 2-D max pooling layer, as