Answers.Body: <p>In general, no.</p>
  
  <p>There is a tradeoff between making the <em>validation set</em> for each fold smaller, and having more folds in total.</p>
  
  <p>As an example, if you have <span class="math-container">$N$</span> folds for <span class="math-container">$N$</span> datapoints, each fold will have only a single datapoint in its validation set. The validation accuracy of a model on a single datapoint is not a reliable estimator for the test performance of the model. In fact, you can construct examples where the error is arbitrarily large.</p>
  
  <p>For this reason, people sometimes use <a href="https://datascience.stackexchange.com/questions/32264/what-is-the-difference-between-bootstrapping-and-cross-validation">Bootstrap Validation</a> if they need a very large number of folds. In practice though, most people just us 10 folds, and that's "good enough". </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-23T15:40:05.737Z
Answers.LastActivityDate: 2019-10-23T15:40:05.737Z
Answers.UserDisplayName: John Doucette
Body: <p>Based on the answer of my previous question:
  <a href="https://ai.stackexchange.com/questions/16038/how-can-i-avoid-overfitting-when-doing-parameter-tuning/16041#16041">How can I avoid overfitting when doing parameter tuning?</a></p>
  
  <p>Can we say: the more we increase the numbers K of cross validation the less likely it is that we overfit?</p>
  
Tags: <cross-validation>
Title: Can we say: the more we increase the numbers of cross validation the less likely it is that we overfit?
UserDisplayName: jennifer ruurs
ThreadText: # Can we say: the more we increase the numbers of cross validation the less likely it is that we overfit?
  
  (Asked by: username_0 on 2019-10-23)
  
  Based on the answer of my previous question:
  [How can I avoid overfitting when doing parameter tuning?](https://ai.stackexchange.com/questions/16038/how-can-i-avoid-overfitting-when-doing-parameter-tuning/16041#16041)
  
  Can we say: the more we increase the numbers K of cross validation the less likely it is that we overfit?
  
  **Tags:** cross-validation
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-10-23. Score: 1 | Accepted answer
  
  In general, no.
  
  There is a tradeoff between making the *validation set* for each fold smaller, and having more folds in total.
  
  As an example, if you have $N$ folds for $N$ datapoints, each fold will have only a single datapoint in its validation set. The validation accuracy of a model on a single datapoint is not a reliable estimator for the test performance of the model. In fact, you can construct examples where the error is arbitrarily large.
  
  For this reason, people sometimes use [Bootstrap Validation](https://datascience.stackexchange.com/questions/32264/what-is-the-difference-between-bootstrapping-and-cross-validation) if they need a very large number of folds. In practice though, most people just us 10 folds, and that's "good enough".
  
  
  ---
  
  