pinazawa123:

My question is the following: Is it correct/fair to apply the WeightedRandomSampler in the TEST and validation datasets? Or it shouldn`t make any difference at all?


I don’t think applying a weighted sampling to the validation and test data would be a valid step.
In my opinion the validation dataset is a proxy of the test dataset and should give you a signal how well your model would perform an new unseen data and can be used for e.g. early stopping.
Once you’ve finished training the model and think the performance is good enough you would run a final iteration on the test dataset to get the model performance on completely new real world data.
If your future task (e.g. when deploying the model) is also dealing with imbalanced data, which should be the case otherwise the test data wouldn’t be imbalanced, I would recommend to leave these datasets as they are.
That being said, you might need to use another metric than the accuracy to measure the performance of your model due to the Accuracy Paradox 26.