I see your point but I guess we can simply calculate the gradient of any tensor right? does not need to be involved directly in calculating the loss right?

Well if the Tensor is not involved in the loss computation, it’s gradient will be 0s everywhere (or None, it is the same thing).
So to have meaningful gradients, you need to use the Tensor, otherwise, they will always be 0 and you won’t be able to learn anything.