Jared_77:

I’m wondering that if when training in batch, is there anyway to avoid the fact that the RNN will take the hidden output of the previous sequence as the input to the first timestep in the next sequence?


The hidden state of the sequence will not be kept in the RNN/LSTM/… modules - you can either pass it in or it will be set to 0s for each invocation. Only the weights are learned.
Best regards
Thomas