I’m training my model on a dataset that I’ve normalized. But I thought that I just had to normalize the train set and not the validation or test sets. However, my validation loss was really inconsistent and it never converged, until I changed the transforms and add the normalized transforms on it. Then the validation loss started to work as expected.
My question is: When I normalize my train set I must normalize my validation and test set as well? If so, why? When I have a single image to predict, will I have to normalized it?