I don’t think resuming the training from a specific epoch forces the loss to decrease, but it sounds more as if your model diverges non-deterministically. I.e. I assume you don’t expect to see bitwise-accurate outputs after resuming the training compared to the initial training run.