Question: I am getting nan score from cross\_val\_score. May I please know how to handle it.. Struggling since a day.

```
from sklearn.model_selection import KFold, cross_val_score
from sklearn.model_selection import train_test_split
import hyperopt
from hyperopt import tpe
from hyperopt import STATUS_OK
from hyperopt import Trials
from hyperopt import hp
from hyperopt import fmin
from sklearn.linear_model import LogisticRegression
df = pd.read_csv('creditcard.csv')
df.head()

def hyperopt_train_test(params):
    cv = StratifiedShuffleSplit(n_splits = 1, test_size = .25, random_state = 0 ) 
    clf =LogisticRegression(**params)

    return cross_val_score(clf,  df.loc[:,:'Amount'], df['Class'],cv = cv,scoring='f1').mean()

space = {
    'C' : hp.uniform('C', 0.05, 10),
    'penalty': hp.choice('penalty',['l2','l1']),
    'max_iter':hp.choice('max_iter',[200,300,400,500])
}

def f(params,scores=[]):
    acc = hyperopt_train_test(params)
    print(acc, "Accuracy")
    scores.append(acc)
    return {'scores':scores,'loss': 1-acc, 'status': STATUS_OK, 'scores':scores}

trials = Trials()
trials
best = fmin(f, space, algo=tpe.suggest, max_evals=10, trials=trials)
print('best:',best)
hyperopt.space_eval(space,best)

```

[DataSet can be downloaded here](https://www.kaggle.com/mlg-ulb/creditcardfraud)

Answer: 
cross\_val\_score method has this weird problem, if there is any issue with the underlying datatypes or parameters and encounters error internally instead of printing the exact error traceback it gives nans in the output.

To understand your specific problem, use this parameter "error\_score"

```
cross_val_score(xgboost.XGBClassifier(), X.values, y.values, cv=kfold, error_score="raise")

```

This will print the exact error which you need to fix. In my case, I had to convert my string variables to floats.
In cross\_val\_score, try using `df.loc[:,:'Amount'].values, df['Class'].values`