Why does XGBoost with cross-validation perform worse on test holdout than unvalidated model? I have an XGBoost model that I fit on some X data directly out of the box:
xgb_outofbox = XGBClassifier(random_state=0).fit(X_tr, y_tr)

This model yields an F1-score of 0.626 on my test holdout data.
Then I build another model:
cv_params = {'learning_rate': [0.1, 0.2, 0.3],
             'max_depth': [2,3,4,5],
             'min_child_weight': [1,2,3,4]
             'n_estimators': [50,75,100,125]
            }
xbg_cv = GridSearchCV(XGBClassifier(random_state=0), cv_params, scoring='f1', cv=5)
xgb_cv.fit(X_tr, y_tr)

This model gives an F1-score of 0.614.
Note: the train_test_split was performed with stratify=y and the entire dataset was shuffled prior to modeling.
I have 2 questions:

*

*Why would the cross-validated model perform worse than the original when it was explicitly set to optimize F1-score?


*The parameters in cv_params include the exact parameters of xgb_outofbox for each item. When I call xgb_cv.best_estimator_, I get different hyperparameters than were used for the out-of-box model. Why wouldn't the cross-validated model settle on the same hyperparameters used by the out-of-box model if they yield a higher F1 score?
All other settings should be the same. I didn't specify anything differently except what I mentioned above.

The cross-validated model performs worse than the "out-of-the-box" model likely because by default max_depth is 6. So when the classifier is fitted "out-of-the-box", we have more expressive base learners. In addition to that, please note that the cross-validated model is not necessarily optimal for a single hold-out test-set. It might be the case that the "out-of-the-box/depth-6" model overfits the training folds during cross-validation so it has a lower CV score than some of the less expressive models (e.g. with "depth-4") but appears better in the hold-out set.
