Hi @googlebot, thank you for your response! 
But theoretically it would also be possible to resume training without the optimizer for a loss in initial performance?
If for example I want to take the pretrained parameters and use them in another network with a different structure. I can do that for the model.state_dict() with the strict=False flag, but there is no such thing for the optimizer.state_dict()…