Good to know, thank you @ptrblck and @albanD for addressing my misconceptions. For others searching for this, my take away from this the discussion is as follows (please correct me if I’m wrong):

There is currently no existing module in the PyTorch library for saving and restoring the model during training. The functionality is there, but It is up to the user to implement how to resume the training of a model.
PyTorch does provide a way to save and restore the model’s parameters through the load_state_dict() method 23.

The same approach works for the optimizer’s gradients, and parameters.


Although serialization methods 11 do exist, they are intended for use with a trained model; or require additional independent logic to save and resume progress during training.
Although there is a torch.utils.checkpoint 6 it is NOT intended for the same use case scenario as TensorFlow’s similarly named checkpoints 2, and is instead used to provide memory traces during computation.