Yes, you could continue training with it, but would most likely get a different loss curve compared to the “full” run without saving and restoring the training (if your optimizer uses internal buffers).
If you load only the model’s state_dict your use case would be similar to the standard fine-tuning approach.