title: Different results between using train_test_split and cross_val_score in sklearn.cross_validation with randomized data
body: <p>I am performing preliminary tests using sklearn in my code.</p>
  
  <p>I am testing:</p>
  
  <p>1) sklearn.cross_validation.cross_val_score</p>
  
  <p>2) sklearn.cross_validation.train_test_split</p>
  
  <p>like in this <a href="https://stackoverflow.com/questions/30364255/difference-between-using-train-test-split-and-cross-val-score-in-sklearn-cross-v">question</a>.</p>
  
  <p>The code is the following:</p>
  
  <pre><code>#X is my data and Y the corresponding binary labels                                                  
  
  #My classifier                                                                     
  clf = svm.SVC(class_weight='auto', kernel=kernel, gamma=gamma,                   
      degree=degree, cache_size=cache_size,probability=probability)                   
  
  
  #1st method: ShuffleSplit and cross validation                                      
  cv = cross_validation.ShuffleSplit(X.shape[0], n_iter=5,                            
      test_size=0.4, random_state=0)                                                  
  #Scoring                                                                            
  scores = cross_validation.cross_val_score(clf, X, Y,                     
      cv=cv, n_jobs=3, scoring="roc_auc")                                             
  
  #2nd method: train_test_split                                                       
  X_train, X_test, y_train, y_test = cross_validation.train_test_split(               
      X, Y, test_size=0.4, random_state=42)                                           
  
  clf.fit(X_train, y_train)                                                          
  pred_test = clf.predict(X_test)                                                     
  #Scoring                                                                            
  score = roc_auc_score(y_test, pred_test)
  </code></pre>
  
  <p>The difference with the other question is that my data is being randomized in both cases 1) and 2).</p>
  
  <p>However I get for case 1) the following scores:</p>
  
  <pre><code>[ 0.9453893   0.94878745  0.95197478  0.95150763  0.94971746]
  </code></pre>
  
  <p>and for case 2):</p>
  
  <pre><code>0.867637
  </code></pre>
  
  <p>I actually quite not understand the reason of this different scores and cannot get what I'm missing here.</p>
  
  <p>Shouldn't the scorings be similar ?</p>
  
  <p>Thank you for your time.</p>