Answers.Body: <p>Before doing the <span class="math-container">$k$</span>-fold cross-val you divide all your dataset into train and test splits (e.g., 80-20 in proportion), then the <span class="math-container">$k$</span>-fold cross validation is performed on the <em>training split</em> as follows:</p>
  <p><a href="https://i.sstatic.net/y6aaM.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/y6aaM.png" alt="enter image description here" /></a>
  The training data is split into <span class="math-container">$k$</span> folds, where at each training iteration a fold is held out for validation: so <span class="math-container">$k-1$</span> folds are effectively used for training. The process repeats <span class="math-container">$k$</span> times, each time using a different fold for validation.</p>
  <p>The <span class="math-container">$k$</span>-fold cross-val approach is useful when your dataset is small, also it can be used as an easy way that provides <span class="math-container">$k$</span> (slightly different) models to build an ensemble (since each iteration trains a new model on a different split of the training data), which is supposed to generalize better. Moreover, since you have <span class="math-container">$k$</span> models you can use them to estimate the variance of their predictions, which can be used as a way to estimate uncertainty.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-08T18:38:36.147Z
Answers.Text: I think it is like this. You divide the dataset into K-folds. 1 fold is used for test. Then, you divide the remaining K-1 folds into training and validation based on a chosen percentage.
Answers.UserDisplayName: DSPinfinity
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-09T22:05:49.867Z
Answers.Text: This is a nice description of k-fold CV, but I don't think this answers the actual question.  Do we need to differentiate between testing a validation datasets in k-fold CV?
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-12T13:58:35.810Z
Answers.Text: @nbro Well, the fact is that in classical ML there is no notion of validation set, which instead is present in modern ML and deep learning. In the first case, you perform K-fold cross-val on the entire dataset without the train-test splitting beforehand: so one fold is used for testing. But in general, especially when you need to tune hyper-parameters, performing k-fold cross-val on the training-split, and after having the real evaluation on the test-set is the safest option to not underestimate the generalization error.
Answers.UserDisplayName: Luca Anzalone
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-08T16:36:30.123Z
Answers.LastActivityDate: 2023-10-08T16:36:30.123Z
Answers.UserDisplayName: Luca Anzalone
Body: <p>We know that in machine learning the dataset is divided into 3 parts: training data, validation data and test data.</p>
  <p>On the other hand, K-fold cross-validation is defined as follows:</p>
  <blockquote>
  <p>the dataset is divided into K number of different sectors. One section is used for testing and the rest for training. The results of these K-iterative tests are then averaged to get the final accuracy.</p>
  </blockquote>
  <p>What happens to validation dataset in K-fold cross-validation? Is there such a dataset?</p>
  
Tags: <machine-learning><comparison><cross-validation><validation-datasets><k-fold-cv>
Title: Is there validation data in K-fold cross-validation?
UserDisplayName: DSPinfinity
ThreadText: # Is there validation data in K-fold cross-validation?
  
  (Asked by: username_0 on 2023-10-08)
  
  We know that in machine learning the dataset is divided into 3 parts: training data, validation data and test data.
  
  On the other hand, K-fold cross-validation is defined as follows:
  
  > the dataset is divided into K number of different sectors. One section is used for testing and the rest for training. The results of these K-iterative tests are then averaged to get the final accuracy.
  
  What happens to validation dataset in K-fold cross-validation? Is there such a dataset?
  
  **Tags:** machine-learning, comparison, cross-validation, validation-datasets, k-fold-cv
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-10-08. Score: 1
  
  Before doing the $k$-fold cross-val you divide all your dataset into train and test splits (e.g., 80-20 in proportion), then the $k$-fold cross validation is performed on the *training split* as follows:
  
  [![enter image description here](https://i.sstatic.net/y6aaM.png)](https://i.sstatic.net/y6aaM.png)
  The training data is split into $k$ folds, where at each training iteration a fold is held out for validation: so $k-1$ folds are effectively used for training. The process repeats $k$ times, each time using a different fold for validation.
  
  The $k$-fold cross-val approach is useful when your dataset is small, also it can be used as an easy way that provides $k$ (slightly different) models to build an ensemble (since each iteration trains a new model on a different split of the training data), which is supposed to generalize better. Moreover, since you have $k$ models you can use them to estimate the variance of their predictions, which can be used as a way to estimate uncertainty.
  
  > **Comments:**
  > **username_0**: I think it is like this. You divide the dataset into K-folds. 1 fold is used for test. Then, you divide the remaining K-1 folds into training and validation based on a chosen percentage.  
  > **username_2**: This is a nice description of k-fold CV, but I don't think this answers the actual question.  Do we need to differentiate between testing a validation datasets in k-fold CV?  
  > **username_1**: @nbro Well, the fact is that in classical ML there is no notion of validation set, which instead is present in modern ML and deep learning. In the first case, you perform K-fold cross-val on the entire dataset without the train-test splitting beforehand: so one fold is used for testing. But in general, especially when you need to tune hyper-parameters, performing k-fold cross-val on the training-split, and after having the real evaluation on the test-set is the safest option to not underestimate the generalization error.  
  
  ---
  
  