Daniel_Dsouza:

[…] bidirectional embedding of a sentence […] they mean they use the final " hn


Yes, I would strongly assume so, since you only get the true/full sentence embedding after the last step.



 Daniel_Dsouza:

[…] using the output[-1] is “forward_final_hidden_layer, backward_initial_hidden_layer”, which is incorrect to use as a sentence representation, correct?


Yes, at least the way PyTorch implements it.
An alternative, I have occasionally stumbled up it, is to calculate the average or maximum over the whole output; please see the forward method of a simple RNN-based classifier I used for testing the effects. It obviously will work (since the dimensions check out) and even train well, but I’m pretty sure I saw any improvements since I stopped using it. Fell free to try, though. I might work for you setting
def forward(self, batch, method='last_step'):
    embeds = self.word_embeddings(batch)
    output = torch.transpose(embeds, 0, 1)
    output, self.hidden = self.gru(x, self.hidden)

    if method == 'last_step':
        output = output[-1] # don't do that for bidirectional=True :)
    elif method == 'average_pooling':
        output = torch.sum(output, dim=0) / len(batch[0])
    elif method == 'max_pooling':
        output, _ = torch.max(x, dim=0)
    else:
        raise Exception('Unknown method.')
        
    for l in self.linears:
        output = l(output)
    log_probs = F.log_softmax(output, dim=1)
    return log_probs