Hi,
First, this part is not really documented more on less on purpose as the backend is moving quite a lot still. Even though we make sure the python/cpp user facing interface do not change, there are still some changes in the backend and what I’m going to describe below is the current state and may not be true forever !
In you example, I guess you miss a requires_grad=True in the x creation, otherwise, no gradient will be computed.
Then in your example:


SB = z.grad_fn is indeed a subclass of Function that corresponds to the backward method that should be called to perform the backward of the sum op that you did.

SB.next_functions is a tuple that contains information of what should be called next. Each entry in the tuple contains another tuple with two elements: The next backward function to call and which output of this other function this input corresponds to. In your case, it should contain a MulBackward0 method for the input and links to the 0th output of the previous function. The input is y and it was created by the multiplication by 2 op. This op had only one output y and so it’s the output 0.

MB = SB.next_functions[0][0] here we get the MulBackward0 Function.

MB.next_functions will contain two things (note that because of the wrapping, the order of arguments here does not match what you wrote in python: x is the first input argument and 2 the second one):

The first argument was created by an AccumulateGrad op and is it’s 0th output.
The second argument has a None backward method as it is the 2 number and no gradients need to flow back to it.



AG = MB.next_functions[0][0] get the AccumulateGrad Function.

AG.variable is x -> True This particular subclass of Function has a .variable attribute that contains the Tensor in which the gradients need to be accumulated. In this case it will contain the Tensor x. And so when calling backward, it will accumulate into the .grad field of the x Tensor.

Final note: Most of the Function subclasses are created by automatically generated cpp code. So it is quite tricky to find them. The file used to generate them can be found here 17, in the link, the backward for the sum is just passing the same gradient value to all the inputs.
I hope this helps