You should check your dataset and ensure there are no singularities in the input.
Just by doing normalization you may find some zero-channel or any other thing
If it’s to big add an epsilon. Another option is keeping a register to check if NaN appears for certain samples always.  You can even put a checker inside the forward (it will be slower but just to debug) which tries to look for NaNs nd infs
Let me know if the dataset is ok
I mean, everyone thinks the data is ok, but you have to be sure it’s truly ok cos a single wrong sample mess everything.
if this sanity check is truly ok, have a look at the batch normalization statistics