Both the encoder and the decoder use a loop in the number of layers, which basically forces the network to share weights across layers. This is not usual, and if it was intended, it could have been explicit in the text.

The number of layer is set to 1, so the loop could be simply removed and it would still work properly.

It could have dealt with attention better. It basically admits that all sentences have the same length, and if they don’t, then they won’t use all attention weights. This basically means that the attention weights used may not sum to 1, as it is supposed.

All the sentences do have the same lengths, after pre-processing the data. But it is free to us to extend this tutorial with variable lengths, in order to explain the pack_padding trick, using an RNN cell for attention (instead of the linear layer). That would be great !