Machine Learning, Model Selection, Hyperparameter Tuning, Learning Curve, Cross Validation.
positive only 10% of the cases. Randomly splitting the data into train/validation/test set according to a 50%/25%/25% split could e.g. lead to a distribution of 5% positive cases in training and 15% in validation & test set, which could seriously screw algorithm performance estimates. In such a