I do agree with you that simply resuming from the checkpoint should not be the reason the loss was decreasing. It didn’t make sense to me either . However, the fact the loss went down after resuming told me the model somehow started to perform better, at least on the training set, right? Are you suggesting the model would only perform better on training set, but not on the test set if I kept paused and resuming the training?