Yes, that would be a classic fine-tuning task and is possible in PyTorch.
As described in the docs you’ve posted, you might also need to save and load the optimizer’s state_dict, if your optimizer has internal states (e.g. Adam uses running estimates).
The Finetuning tutorial 224 explains how to load pre-trained torchvision models and fine-tune them. Instead of loading the torchvision model, you might just want to load your custom model and adapt the code appropriately.