title: Create training and test dataset in Pandas
body: <p>I would like to manually create a training and test dataset in pandas rather than use cross-validation from sklearn. I am nearly successful. However, I find discrepancy in the numbers between <strong>df_training</strong> and <strong>df_test</strong>. Why is that? </p>
  
  <p>Here's what I did: </p>
  
  <ul>
  <li>Created a new dataset called <strong>df_training</strong> from original data frame by selecting rows at random with random.choice</li>
  <li>Created a new dataset called <strong>df_test</strong> by dropping rows from the original dataset that are in df_training using the df.drop(df_training.index) as the argument. </li>
  </ul>
  
  <p>This is where I do not get the correction dimensions of df_test when the dimensions of df and df_training remain constant.  </p>
  
  <pre><code>from sklearn.datasets import load_boston 
  boston = load_boston()
  
  names = ['crim', 'zn', 'indus', 'chas', 'nox', 'rm', 'age', 'dis', 'rad', 'tax', 'ptratio', 'b', 'lstat']
  df = pd.DataFrame(boston.data, columns=names)
  # add in prices
  df['price'] = boston.target
  
  df.shape
  (506, 14)
  
  import random
  # Use 70% of the DataFrame and call is df_training
  df_training = df.ix[np.random.choice(df.index, 354)]
  df_training.shape
  
  # Remove the 70% of data from the main DataFrame and call it df_test
  df_test = df.drop(df_training.index)
  
  df_test.shape
  (250, 14)
  </code></pre>
  
  <p>Should I not get 504 - 354 = 150?</p>
  
  <p>Interestingly when I run whole code couple of times I get different results for the test_set. Shouldn't I get the same result when training set and original set are constant? What's going on? </p>
  
  <pre><code>In [26]: %run create_training.py
  Original Set:  (506, 14)
  training set:  (354, 14)
  test set:  (247, 14)
  
  In [27]: %run create_training.py
  Original Set:  (506, 14)
  training set:  (354, 14)
  test set:  (254, 14)
  
  In [28]: %run create_training.py
  Original Set:  (506, 14)
  training set:  (354, 14)
  test set:  (241, 14)
  </code></pre>