Hi! I am facing a strange situation. I am using Resnet 101 which contains batchNorm layers for segmentation. The batch size while testing is set to 1. The output is better if I use train() mode as compared to eval() mode. According to the batchnorm formula, when batch size is 1, mean[x]=x and Var[x]=0. Thus the final output of the formula will be beta alone (gamma multiplied by 0 is 0). This should give very bad results as the input information is not propagated by the batchNorm layer. Surely, I am making some mistake. Any help is appreciated.
Regards,
Omkar Damle.