Solved by ptrblck in post #2 
              
              
                Could you try to save the state_dict instead of the model and optimizer directly? 
Then while restoring, try to use strict=False in .load_state_dict.