I actually meant to say to only add the BatchNorm as the first layer of your network (so before conv1) in order to “simulate” normalisation. But if this works and avoids the NaN then indeed your problem (or part of it) seems to be normalisation or more correct the lack of it.
The downside of BatchNorm is that the normalisation only happens per batch, so 64 images in your case. You normally achieves better results if you do the normalisation based on your complete training set and not just 64 at a time.
So you calculate the mean and variance for all you training images and then normalise each image using this mean and variance. This would be done in the preprocessing phase (and not part of your network as is the case with BatchNorm).I’m guess some searching on image normalisation should give you some reusable code snippets.