Solved by KFrank in post #3 
              
              
                Hi Yongjie! 

Your process it mostly correct.  You need to use retain_graph = True 
in your first backward() call: 
loss1.backward (retain_graph = True)


No, one forward pass is enough.  (You do, of course, have to calculate 
both loss1 and loss2.)  But you do need two backward() passes. 
(If you w…