title: How to handle multiclass with Stratified Cross Validation in sklearn
body: <pre><code>from sklearn.model_selection import RandomizedSearchCV, GridSearchCV
  from sklearn.metrics import roc_auc_score
  from sklearn.metrics import precision_recall_fscore_support as score
  from sklearn.model_selection import StratifiedKFold
  from xgboost import XGBClassifier
  import time
  
  params = {
      'min_child_weight': [1, 5, 10],
      'gamma': [0.5, 1, 1.5, 2, 5],
      'subsample': [0.6, 0.8, 1.0],
      'colsample_bytree': [0.6, 0.8, 1.0],
      'max_depth': [3, 4, 5]
      }
  
  
  
  xgb = XGBClassifier(learning_rate=0.02, n_estimators=600,
                  silent=True, nthread=1)
  
  folds = 5
  param_comb = 5
  
  skf = StratifiedKFold(n_splits=folds, shuffle = True, random_state = 1001)
  
  random_search = RandomizedSearchCV(xgb, param_distributions=params, n_iter=param_comb, scoring=['f1_macro','precision_macro'], n_jobs=4, cv=skf.split(X_train,y_train), verbose=3, random_state=1001)
  
  start_time = time.clock() # timing starts from this point for "start_time" variable
  random_search.fit(X_train, y_train)
  elapsed = (time.clock() - start) # timing ends here for "start_time" 
  variable
  </code></pre>
  
  <p>My code is above, my y_train is a pandas series with multiclass with integers from 0 to 9. </p>
  
  <pre><code>y_train.head()
  1041    8
  1177    7
  2966    0
  1690    2
  2115    1
  Name: Industry, dtype: object
  </code></pre>
  
  <p>Once I run the setup code above, I received error message as:</p>
  
  <pre><code>ValueError: Supported target types are: ('binary', 'multiclass'). Got 'unknown' instead.
  </code></pre>
  
  <p>I did some search on other similar questions, I tried to use <code>cross_validate</code> from   <code>sklearn.model_selection</code> and tried to use other metrics compatible with multiclass but still got the same error message. </p>
  
  <p>Is there anyway I can do gridsearch on parameters with stratified cross validation based on performance metrics?</p>
  
  <p>Update: After fixing the <code>dtype</code> issue, I want to pass multiple metrics into <code>scoring=</code>, I tried in this way because I read this document (<a href="http://scikit-learn.org/stable/modules/model_evaluation.html#scoring-parameter" rel="nofollow noreferrer">http://scikit-learn.org/stable/modules/model_evaluation.html#scoring-parameter</a>):</p>
  
  <pre><code>random_search = RandomizedSearchCV(xgb, param_distributions=params, n_iter=param_comb, scoring=['f1_macro','precision_macro'], n_jobs=4, cv=skf.split(X_train,y_train), verbose=3, random_state=1001)
  </code></pre>
  
  <p>Then I failed with following warning:</p>
  
  <pre><code>ValueError                                Traceback (most recent call 
  last)
  &lt;ipython-input-67-dd57cd97c89c&gt; in &lt;module&gt;()
   36 # Here we go
   37 start_time = time.clock() # timing starts from this point for 
  "start_time" variable
  ---&gt; 38 random_search.fit(X_train, y_train)
   39 elapsed = (time.clock() - start) # timing ends here for "start_time" variable
  
  /anaconda3/lib/python3.6/site- 
  packages/sklearn/model_selection/_search.py in fit(self, X, y, groups, 
  **fit_params)
  609                                  "available for that metric. If 
  this is not "
  610                                  "needed, refit should be set to 
  False "
  --&gt; 611                                  "explicitly. %r was passed." % 
  self.refit)
  612             else:
  613                 refit_metric = self.refit
  
  ValueError: For multi-metric scoring, the parameter refit must be set 
  to a scorer key to refit an estimator with the best parameter setting 
  on the whole data and make the best_* attributes available for that 
  metric. If this is not needed, refit should be set to False explicitly. 
  True was passed.
  </code></pre>
  
  <p>How to fix this issue?</p>