Does the validation data come from approx. the same domain as the train data, e.g. are both “natural” images?
Are you preprocessing the validation data in the same way as the train data, e.g. normalization etc.?