title: Interpreting sklearns' GridSearchCV best score
body: <p>I would like to know the difference between the score returned by <code>GridSearchCV</code> and the <code>R2</code> metric calculated as below. In other cases I receive the grid search score highly negative (same applies for <code>cross_val_score</code>) and I would be grateful for explaining what it is.</p>
  
  <pre><code>from sklearn import datasets
  from sklearn.model_selection import (cross_val_score, GridSearchCV)
  from sklearn.tree import DecisionTreeRegressor
  from sklearn.metrics import accuracy_score, r2_score
  from sklearn import tree
  
  diabetes = datasets.load_diabetes()
  X = diabetes.data[:150]
  y = diabetes.target[:150]
  X = pd.DataFrame(X)
  
  parameters = {'splitter':('best','random'), 
                'max_depth':np.arange(1,10), 
                'min_samples_split':np.arange(2,10), 
                'min_samples_leaf':np.arange(1,5)}
  
  regressor = GridSearchCV(DecisionTreeRegressor(), parameters, scoring = 'r2', cv = 5)
  regressor.fit(X, y)
  
  print('Best score: ', regressor.best_score_)
  best = regressor.best_estimator_
  print('R2: ', r2_score(y_pred = best.predict(X), y_true = y))
  </code></pre>