Solved by ptrblck in post #4 
              
              
                I agree with @KFrank also also think that the model, criterion, and optimizer are separate units, which can be independently used and changed. 
E.g. reusing your “model” to finetune it in another scenario would also recreate your optimizer, which would not be needed. Restoring a checkpoint also woul…