That’s a good point I have been wondering about too.
According to the LSTM doc:
output of shape (seq_len, batch, num_directions * hidden_size): tensor containing the output features (h_t) from the last layer of the LSTM, for each t.
To my understanding, for the forward pass, we want -1 which is the last step T.
For the backward pass, we want the step 0 which is the last reverse step.
Maybe someone around the forum can confirm whether torch.cat([E_output[:,-1,0,:], E_output[:,0,1,:]],-1) does that or not ?