@ptrblck, Thank you for the clarification, I am really sorry to keep troubling you with a barrage of questions . I actually have a few more doubts about BatchNorm layers in PyTorch and its behavior. So can we have that discussion in this post itself or is there any specific post (I have found quite a few about BatchNorm!) I can post my questions on?
Or should I open a new topic altogether? Thanks!