Because it’s a running_mean it’s updated as an moving average between the current estimate of the mean and the old value rather than being done via 2 tunable parmeters. It’s explained within the docs BatchNorm1d — PyTorch 1.9.1 documentation