Yeah you are completely right! Better not to normalize the whole batch!
I took a look to my code and the normalization is being done in a subclass of torch.utils.data.Dataset. So you are right. It is better just to normalize by sample.