Thanks for the responses so far.
Hopefully this makes my initial question clear:  Say you just want to normalize to 0 mean.  The operation is y = (x - mu).
Is the ‘batchnorm equivalent’ approach like:
y = x - x.mean(dim=0).detach()

or
y = x - x.mean(dim=0)

While the value of y is the same, the gradient propagation is clearly different.