I have saved and loaded the checkpoints as per the pytorch manual, and it all seems OK. Now, usually, when I want to start training, I have something like this in pytorch:
    for itr in range(1, args.niters + 1):
        optimizer.zero_grad() # should I or should I not?

I am unsure if I should do zero_grad() here (which I use when I start training from scratch), since I am reloading all my weights and bias.
Apologies if this is a daft question.