Thanks for the follow up. The new experiment seems to indicate that the issue is not in the batchnorm layer, but apparently the general training.
I cannot see any obvious errors in the code, so I would recommend to try to overfit the model on a small data subset first (e.g. just 10 sample) and make sure the model is able to do so.
If that’s not the case, there might be a bug I haven’t seen in the code snippets or in another part of the code, which is not posted.