Answers.Body: <p>When training a ML or DL model it's crucial to have a separate <a href="https://en.wikipedia.org/wiki/Training,_validation,_and_test_data_sets#Validation_data_set" rel="nofollow noreferrer">validation data set</a> to tune hyperparameters and prevent overfitting. Validation data set should come from the original training data which is divided into a new training data set and a validation data set.</p>
  <blockquote>
  <p>A validation data set is a data set of examples used to tune the hyperparameters (i.e. the architecture) of a classifier. It is sometimes also called the development set or the &quot;dev set&quot;... In order to get more stable results and use all valuable data for training, a data set can be repeatedly split into several training and a validation data sets. This is known as cross-validation. To confirm the model's performance, an <em>additional</em> test data set held out from cross-validation is normally used.</p>
  </blockquote>
  <p>The original test data set is used to evaluate the final model performance on unseen data and usually we keep the test data set separate and only use it for the final evaluation of the model after all training and validation steps are completed. Using part of the test set for validation would mean that the test set is no longer completely unseen, leading to potential data leakage and biased performance metrics.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-03T22:08:37.773Z
Answers.LastActivityDate: 2024-08-03T22:08:37.773Z
Answers.UserDisplayName: cinch
Answers.Body: <p>Total Dataset = (Training set + Validation set ) + Test set</p>
  <p>Typical dataset Ratios:</p>
  <p>Training set : 60-80%
  Validation set : 10-20%
  Test Set : 10-20%</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-07T02:30:06.950Z
Answers.LastActivityDate: 2024-08-07T02:30:06.950Z
Answers.UserDisplayName: Biku
Body: <p>To train a deep learning model, I work with a dataset that is divided into train and test parts by constructors.</p>
  <p>I'm stuck on how to select some data for validation? From the train part or from the test part?</p>
  <p>It seems that dividing the test part into a validation and a new smaller test part is a good idea; but I'm not sure.</p>
  <p>Any hint is appreciated.</p>
  
Tags: <deep-learning><validation-datasets><validation>
Title: How to choose validation data?
UserDisplayName: user153245
ThreadText: # How to choose validation data?
  
  (Asked by: username_0 on 2024-08-03)
  
  To train a deep learning model, I work with a dataset that is divided into train and test parts by constructors.
  
  I'm stuck on how to select some data for validation? From the train part or from the test part?
  
  It seems that dividing the test part into a validation and a new smaller test part is a good idea; but I'm not sure.
  
  Any hint is appreciated.
  
  **Tags:** deep-learning, validation-datasets, validation
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2024-08-03. Score: 1 | Accepted answer
  
  When training a ML or DL model it's crucial to have a separate [validation data set](https://en.wikipedia.org/wiki/Training,_validation,_and_test_data_sets#Validation_data_set) to tune hyperparameters and prevent overfitting. Validation data set should come from the original training data which is divided into a new training data set and a validation data set.
  
  > A validation data set is a data set of examples used to tune the hyperparameters (i.e. the architecture) of a classifier. It is sometimes also called the development set or the "dev set"... In order to get more stable results and use all valuable data for training, a data set can be repeatedly split into several training and a validation data sets. This is known as cross-validation. To confirm the model's performance, an *additional* test data set held out from cross-validation is normally used.
  
  The original test data set is used to evaluate the final model performance on unseen data and usually we keep the test data set separate and only use it for the final evaluation of the model after all training and validation steps are completed. Using part of the test set for validation would mean that the test set is no longer completely unseen, leading to potential data leakage and biased performance metrics.
  
  
  ---
  
  ### Answer by username_2 on 2024-08-07. Score: 0
  
  Total Dataset = (Training set + Validation set ) + Test set
  
  Typical dataset Ratios:
  
  Training set : 60-80%
  Validation set : 10-20%
  Test Set : 10-20%
  
  
  ---
  
  