If I set a seed with (torch.manual_seed(n)) before I iter my dataloader in every epoch I have ensured that my randomness is the same if i restore the training at a given epoch step. After every epoch i generate a new seed and then I just have to save that seed when I stop my training within an epoch.