I am using following as reference. It appears there are two main ways to save the model:
https://pytorch.org/tutorials/beginner/saving_loading_models.html

save dictionary state: recommended way.
save entire model: not recommended as path is saved and can break if it is opened somewhere else.

TF can conveniently store in h5 format which I can dump quickly using h5 (hdf5) format, but this is not possible with Pytorch?