In the encoder above the sequence is run through all at once, so the bidirectional part works.
I have found passing just the forward half of the hidden to work well, decoder_hidden = encoder_hidden[:decoder.n_layers]


There’s a (not totally finished) version of the batched seq2seq model here: https://github.com/spro/practical-pytorch/blob/master/seq2seq-translation/seq2seq-translation-batched.ipynb 321