Hi,
The usual suspects here are the accumulation of some training statistics like the loss that is missing a .detach() or a .item() and for RNN models, the hidden layers that are not reseted properly (still some link to the past) between runs.