i’m not an expert on this but i think that when you train a batch it all gets forwarded at the same time and the hidden states of one sequence in the batch is not used as the input to the next sequence. that’s why the hidden states are of size “lstm_layers X batch size X hidden size”