It is probably because you are using LSTMs. Pytorch’s implementation returns to you both h_n and c_n (hidden state and cell state for the last time step) in the hidden variable as a tuple. In comparison, GRU would just return to you h_n. As a result for LSTMs, hidden[0] is giving you h_n.