<p>In general, no.</p>

<p>There is a tradeoff between making the <em>validation set</em> for each fold smaller, and having more folds in total.</p>

<p>As an example, if you have <span class="math-container">$N$</span> folds for <span class="math-container">$N$</span> datapoints, each fold will have only a single datapoint in its validation set. The validation accuracy of a model on a single datapoint is not a reliable estimator for the test performance of the model. In fact, you can construct examples where the error is arbitrarily large.</p>

<p>For this reason, people sometimes use <a href="https://datascience.stackexchange.com/questions/32264/what-is-the-difference-between-bootstrapping-and-cross-validation">Bootstrap Validation</a> if they need a very large number of folds. In practice though, most people just us 10 folds, and that's "good enough". </p>

CC BY-SA 4.0
2019-10-23T15:40:05.737Z
2019-10-23T15:40:05.737Z
John Doucette
<p>Based on the answer of my previous question:
<a href="https://ai.stackexchange.com/questions/16038/how-can-i-avoid-overfitting-when-doing-parameter-tuning/16041#16041">How can I avoid overfitting when doing parameter tuning?</a></p>

<p>Can we say: the more we increase the numbers K of cross validation the less likely it is that we overfit?</p>

<cross-validation>
Can we say: the more we increase the numbers of cross validation the less likely it is that we overfit?
jennifer ruurs
# Can we say: the more we increase the numbers of cross validation the less likely it is that we overfit?

(Asked by: username_0 on 2019-10-23)

Based on the answer of my previous question:
[How can I avoid overfitting when doing parameter tuning?](https://ai.stackexchange.com/questions/16038/how-can-i-avoid-overfitting-when-doing-parameter-tuning/16041#16041)

Can we say: the more we increase the numbers K of cross validation the less likely it is that we overfit?

**Tags:** cross-validation


---

## 1 answers

### Answer by username_1 on 2019-10-23. Score: 1 | Accepted answer

In general, no.

There is a tradeoff between making the *validation set* for each fold smaller, and having more folds in total.

As an example, if you have $N$ folds for $N$ datapoints, each fold will have only a single datapoint in its validation set. The validation accuracy of a model on a single datapoint is not a reliable estimator for the test performance of the model. In fact, you can construct examples where the error is arbitrarily large.

For this reason, people sometimes use [Bootstrap Validation](https://datascience.stackexchange.com/questions/32264/what-is-the-difference-between-bootstrapping-and-cross-validation) if they need a very large number of folds. In practice though, most people just us 10 folds, and that's "good enough".


---

