You might want to read the doc to get all the details: BatchNorm2d — PyTorch 1.8.0 documentation 9
But the short version is that during training, it computes the stats on the current batch and use that to compute the output and update the running stats.
During evaluation, it uses the saved running stats to compute the output.