Solved by ptrblck in post #8 
              
              
                Even though you are calculating the center manually, does Autograd need to backpropagate through these operations? 
Compare it to the target of a classification use case. While the target tensor is of course not a constant value, it’s a constant in the sense that Autograd will use it to calculate th…