Do you use any ImageNet pretrained network? If answer is Yes, then you should normalize your input images with ImageNet stats.
If answer is No, then you should not use ImageNet stats, instead you should use the stats based on your training images.
Let’s say you have 10.000 training images and 1.000 validation images.
You may calculate the mean and standard deviation for your 10.000 images for all 3 channels if you have 3 channels and you should use that for both the training and validation set to normalize the images.
It won’t be wrong to use all 11.000 images and to calculate the mean and std and use that.
In fact it would be even better.
But once you get these numbers you need to stick with them.