Solved by albanD in post #4 
              
              
                I’m sorry I don’t understand this.  autograd.grad  is does forward differentiation, doesn’t it? 

No it does backward mode as well. Only that instead of populating the .grad field of all the leaf Tensor, it computes and returns the gradients for a list of Tensor that is given. 

My thinking was tha…