Solved by jpeg729 in post #2 
              
              
                deepcopying the optimiser won’t work because it will either give you another optimiser for model1, or more likely simply break the references to the model parameters. 
Here is what I would do… 
model2 = Model() # get new instance
model2.load_state_dict(model1.state_dict()) # copy state
opt2 = torch.…