Solved by ptrblck in post #2 
              
              
                The code looks generally alright. 
If you don’t want loss2.backward() to calculate gradients in model1, you could detach output1 before passing it to model2 via: 
output2 = model2(output1.detach())

This will make sure to cut the computation graph at this point.