Solved by albanD in post #2 
              
              
                Hi, 
There is no difference between the two in the gradients that are computed. 
Because of some optimization we do (to reduce memory usage), you might have to pass retain_graph=True to the first backward to avoir errors.