@vdw Thanks for reminding me. It doesn’t make much sense to combine the last hidden state of the forward pass with the first hidden state of the backward pass in sequence classification tasks.
@moorccini In the bidirectional case, output[-1, :, :hidden_size] gives you the last hidden state of the forward pass, and output[0, :, hidden_size:] gives you the last hidden state of the backward pass. Then you can concat these states/transform them as desired. But I like @vdw’s solution better, since it seems more elegant. If you just pass the output[-1] (which has the first hidden state of the backward pass) of the bidirectional model to the next layer, in a sequence classification task, the model performance could be even worse compared to the unidirectional model. The output shape makes the output suitable for sequence labelling, so it can be a bit tricky to get it right in sequence classification tasks.