I’m not sure, how Tensorflow is implemented in the backend, but note that in PyTorch also backward functions are implemented for you.
Autograd will call these functions, so that you don’t have to manually write them.