tmaric:

Can you point me to a source of documentation beyond what I’ve linked, that describes how exactly autograd::grad works?


autograd.grad does reverse mode automatic differentiation (also called backpropagation), between the output and input that are given, using grad_output as first gradient (which default to 1 for scalar outputs).
You can find some work in progress doc here: Advanced autograd — PyTorch master documentation 1

Should I go into the source code?

I don’t think that will help as it involves quite a lot of components and they interact in non obvious ways

so if x in R^3, partial_x (dot(x,x))=2x. 

Yes that works as expected.
Note that for a first example, I would avoid re-using the same variable as it introduces implicit accumulations in the backward pass.