Assuming that I am fine-tuning a pre-learned model trained with running statistics.
Based on BatchNorm2d — PyTorch 1.7.0 documentation 1
image1012×259 21.9 KB
If momentum = 0,  the running statisics X_new should be equal to the pre-trained statistics X_hat, and therefore we should be able to obtain the same results for both Setting 1 and 2?