I replace nn.BatchNorm2d with MyBN in Resnet18 and use Cifar100 for training.
and it turned out that the loss would be extremely high (e.g. 500) or even nan, and the acc is approximately 1%.
while in the nn.BatchNorm2d case, I could witness the decline of loss and the rise of accuracy.