Hi,
This is actually never built explicitly.
What happens is that when the functions of the forward are seen, they corresponding backward ops are recorded.
And so if you say y = f( g(x) ) where g is the square function and f is time 5, noting z = g(x), what the backward computes is dy/dx as dy/dz * dz/dx = df(z)/dz * dg(x)/dx = 5 * 2x. Note that the 2x is computed first by the backward of g then 5 times that result for the backward of f.
Basically pytorch is just doing backprop, each elementary functions one at a time. Nothing more fancy.