Solved by tom in post #2 
              
              
                So what do you want to do for the parameters that appears in both? 
If you want to update them twice, you don’t really need to do anything except keeping the zero_grad-backward-step bits of each model disentangled. For the first, you probabably want retain_graph or so. 
An alternative could be to up…