Solved by albanD in post #2 
              
              
                I assume there is a typo in your example and model2 is the one used to generate out2 right? 
If so, if you a single optimizer, the best way I can see is: 
model2.zero_grad()
loss2.backward(retain_graph=True)
model0.zero_grad(); model1.zero_grad()
(loss1 + loss3).backward()

opt.step()