Solved by albanD in post #4 
              
              
                Hi, 
The difference is that out.backward() will compute the gradient for all the leaf Tensors that were used to compute out and accumulate these gradients in their .grad  field. 
autograd.grad(out, inp) will compute the gradient of out wrt inp and return that gradient directly. 
One way to see it is…