@dpernes, Thank you for your reply.
I have also think of these that zeros by ReLU can lead to division by zero. But then I have checked the PyTorch implementation of BatchNorm1d 4, and I can see that they have added eps to variance to overcome this.