If I understand it correctly, you have two datasets taken in different conditions, and you want your model to apply well to both of them?
If you believe that the model should apply well to both (the subjects are similar, for instance) but the conditions they were taken in are interfering, I can see two main options:

If you can know in advance if any unseen data will come from a good source or bad source (picture metadata etc., image statistics), then you can preprocess it to resemble Dataset 1.
If you can’t predict this or run such preprocessing, then you can combine the two datasets and normalize them, then train on the combined dataset. This will, however, probably have worse accuracy than a model trained only on Dataset 1 or only on Dataset 2 when tested on those.




 paraujo:

Hi @KFrank , thanks for the explanation. If I correctly understand, if I train a model using dataset 1 and test it in dataset 2, which is a similar dataset, the model should generalize to a new dataset, correct?
When I test the model in the second dataset, the results are quite wrong. Is it an indication that the dataset is wrong or the model lacks training?


If the second dataset is sufficiently similar to the first, then the hope is that a model trained on the first dataset will generalise. If the results are wrong, then it might mean that the model has too much training and has overfit your first dataset (it doesn’t generalise), it might mean the second dataset isn’t similar enough to the first (it would generalise to unseen data, but only if it’s actually similar), or it could mean that the second dataset hasn’t been correctly processed (the model would generalise and the dataset is similar, but the format is off).
You could combine the two datasets into one and train a single model on them, but at that point your test set has become part of your training and validation set and you need new test data–but if it represents a large category you want to generalise to with or without additional image preprocessing, it might be worth it rather than having a narrowly specialised model.