Machine Learning, Transformers, Ashish Vaswani, Lstm, Recurrent Neural Network.
encoder(left), we compute the representation for every position simultaneously using attention. We have feed-forward layers and residual connections between the inputs and outputs of this block. In the decoder(right), we mimic a language model using self-attention. Thus, a position should not be