@spro  thanks for your answer!
But I have another questions,

The first output of encoder with bidirectional rnn should depends on the first and the whole input sequence in reverse order. However, the encoder input is fed word by word, before the encoder output the first output, it should only see the first word of the input sequence. The other words aren’t fed in the encoder yet. Do I misunderstand something?
Due to the bidirectional rnn, by taking n_layer=1, the last hidden state is in the shape of (2, batch_size, hidden_size). But the rnn in decoder is not bidirectional, so the hidden state which fed into decoder should be in the shape of (1, batch_size, hidden_size). So what should be done to the hidden state of the encoder? Should it be

hidden = hidden[0, :, :] + hidden[1, :, :]

Thanks!!