Machine Learning, Model Selection, Hyperparameter Tuning, Learning Curve, Cross Validation.
hold-out fold. For each set of hyperparameters we compute the mean validation score and select the hyperparameter set with best performance on the hold-out validation sets. Alternatively, we can apply the “one-standard-error-rule” [2], which means that we choose the most parsimonious model (the