Thanks, Chris, for your answer.
As far as I understood from the theory of rnn/lstm, that for each element of a sequence there will be hidden state and at the end of the sequence there will be a final hidden state. Also, there is another state of lstm other than the hidden state which is called cell state.
In pytorch, lstm will return two things. The first one is hidden states of each element of the sequences, the variable out should hold this in my case. The second will hold final hidden states of each sequence along with the cell state. In my example, the hidden variable should hold these and hidden[0] should hold the sequence final states and hidden[1] should hold the cell state.
So, if there are two sequences in a mini-bach, should not be there 2 final hidden states hold by variable hidden[0] ?