title: Evaluating estimator performance in scikit-learn
body: <p>I'm using scikit-learn 0.13.1 for a contest on <a href="https://www.kaggle.com" rel="nofollow noreferrer">Kaggle</a>.  I'm using a Decision Tree classifier, and to evaluate my estimator I follow the techniques of splitting the training data via train_test_split, or doing cross validation using cross_val_score.  Either technique would show that the estimator is about 90% accurate.  However, when I use the estimator on actual test data, the accuracy obtained is about 30%  lower.  Let's assume that the training data is a good representation of the test data.</p>
  
  <p>What else can I do to evaluate the accuracy of the estimator?</p>
  
  <pre><code>clf = tree.DecisionTreeClassifier( )
  ...
  X_train, X_test, y_train, y_test = train_test_split(train, target, test_size=0.3, random_state=42)
  ...
  clf.fit(X_train, y_train)
  print "Accuracy: %0.2f " % clf.score(X_test, y_test)
  ...    
  scores = cv.cross_val_score(clf, train, target, cv=15)
  print "Accuracy: %0.2f (+/- %0.2f)" % (scores.mean(), scores.std() / 2)
  </code></pre>