Solved by albanD in post #2 
              
              
                Hi, 
If I understand properly, you have this: 
input = # ...
out1 = Model1(input)
out2 = Model2(out1)
loss = LossFunc(out2)

If you want to optimize the parameters of Model1, you can just use loss.backward() and create your optimizer to only update the first model with optimizer = torch.optim.SGD(Mo…