Thank you Thomas. Just to make sure, in the case of my example:
hidden_dim = 200
embedding_dim = 10
vocab_size = 6
ids = torch.tensor([1,5,4])
bs = 1
embedding = nn.Embedding(vocab_size,embedding_dim)
lstm  = nn.LSTM(embedding_dim, hidden_dim,num_layers=1, bidirectional=True)
embeds = embedding(ids)
output,_ = lstm(embeds)

output[0,1,:] contains the hidden states generated just after both forward and backward LSTMs consume “1”, right ?