Solved by ptrblck in post #2 
              
              
                Both approaches should compute the same gradients. 
In the second approach you would need to call lossGen.backward(retain_graph=True), otherwise the intermediate values will be cleared and you’ll get an error calling lossSeg.backward(). 
However, currently you are using lossSeg to calculate gradient…