BatchNorm will standardize the data by default using the mean and variance (or running mean and running variance during eval). The affine parameters gamma and beta (weight and bias in nn.BatchNorm2d, respectively) allow this layer to undo the standardization or to manipulate it in any way beneficial for the model and training.
E.g. if standardizing the activation might not be beneficial for the training, gamma might be pushed towards the value of the standard deviation, and beta might be learned to be close to the mean value of the data, which will eventually create an identity layer.
Let me know, if this makes things clearer or if I’ve missed some points.