Even though you are calculating the center manually, does Autograd need to backpropagate through these operations?
Compare it to the target of a classification use case. While the target tensor is of course not a constant value, it’s a constant in the sense that Autograd will use it to calculate the loss, and based on this compute the gradient for the parameters in the model, which created the prediction.
Would the same use case be applied here?