Answers.Body: <p>You can cluster all your features in one matrix X, in which each line would be one element of the data set you want to construct, and each column would be a different feature of this element.</p>
  
  <p>You construct then a Y vector containing the different target classes, where the i-th element will be the target class of the i-th X element.</p>
  
  <p><strong><em>For the following I suppose you use python</em></strong></p>
  
  <p>Create train and test sets using sklearn wrapper <strong>train_test_split</strong> on X and Y just built, in which you can shuffle your X and Y (non-avoidable in this case) : </p>
  
  <pre><code>from sklearn.model_selection import train_test_split
  </code></pre>
  
  <p>BTW, I recommend you to rescale your data with sklearn with for example one of :</p>
  
  <pre><code>from sklearn.preprocessing import MinMaxScaler, StandardScaler
  </code></pre>
  
  <p>It allows better learning and so better generalization</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-03-21T02:09:53.763Z
Answers.Text: Well done on train and test split and scaling. However, @stain also asked about validation set. Some packages (eg most of Keras supervised learning algorithms) can consider a portion (by with absolute or relative size) of training set as the validation set. Otherwise, one can perform `train_test_split` twice (one on whole data and once on the train set). See https://datascience.stackexchange.com/a/15136/46505
Answers.UserDisplayName: Borhan Kazimipour
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-03-21T10:46:24.120Z
Answers.Text: I also thought of mentioning cross-validation using k-splits (kfolds) generalizing the concept of training. As it is mentioned in your link @O_o answer seems also relevant here.
Answers.UserDisplayName: nsaura
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-03-20T14:56:28.603Z
Answers.LastActivityDate: 2018-03-20T15:03:28.107Z
Answers.LastEditDate: 2018-03-20T15:03:28.107Z
Answers.UserDisplayName: nsaura
Body: <p>Right now, I'm planning to make a deep neural network for classifying taste of crystals, with their molecular structure, which includes information like the number of atoms or the mass of each atom.</p>
  <p>How should I make a data set for training, testing, and validation?</p>
  
Tags: <machine-learning><classification><datasets>
Title: In order to classify the taste of crystals, how should I make the training, validation and test datasets?
UserDisplayName: STAIN
ThreadText: # In order to classify the taste of crystals, how should I make the training, validation and test datasets?
  
  (Asked by: username_0 on 2018-03-20)
  
  Right now, I'm planning to make a deep neural network for classifying taste of crystals, with their molecular structure, which includes information like the number of atoms or the mass of each atom.
  
  How should I make a data set for training, testing, and validation?
  
  **Tags:** machine-learning, classification, datasets
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-03-20. Score: 2
  
  You can cluster all your features in one matrix X, in which each line would be one element of the data set you want to construct, and each column would be a different feature of this element.
  You construct then a Y vector containing the different target classes, where the i-th element will be the target class of the i-th X element.
  For the following I suppose you use python
  Create train and test sets using sklearn wrapper train_test_split on X and Y just built, in which you can shuffle your X and Y (non-avoidable in this case) : 
  from sklearn.model_selection import train_test_split
  
  BTW, I recommend you to rescale your data with sklearn with for example one of :
  from sklearn.preprocessing import MinMaxScaler, StandardScaler
  
  It allows better learning and so better generalization
  
  > **Comments:**
  > **username_2**: Well done on train and test split and scaling. However, @stain also asked about validation set. Some packages (eg most of Keras supervised learning algorithms) can consider a portion (by with absolute or relative size) of training set as the validation set. Otherwise, one can perform `train_test_split` twice (one on whole data and once on the train set). See https://datascience.stackexchange.com/a/15136/46505  
  > **username_1**: I also thought of mentioning cross-validation using k-splits (kfolds) generalizing the concept of training. As it is mentioned in your link @O_o answer seems also relevant here.  
  
  ---
  
  