Answers.Body: <p><strong>Purely in terms of overfitting</strong>, and assuming you train both for equal amounts of time, 70/30 is probably better but performance is not going to be very good. Not training on %30 of data will make both training and test results equally bad (in my opinion). But it won't overfit, that is for sure. Cross validation (you have in mind 90/10, I assume) will take a long time, so that won't have enough time to train and it might be overfitting more compared to 70/30, but as it is going to see all training samples %90 at a time, there is a good chance it will train better. So, at the end of the day, it will overfit more but perform better. </p>
  
  <p>If you are asking which is better overall, performance and overfitting, I say it depends on the size of your dataset. If you have millions of samples in it, you can even use a 98/1/1 for training, testing and validation and <a href="https://www.coursera.org/learn/deep-neural-network/lecture/cxG1s/train-dev-test-sets" rel="nofollow noreferrer">still be OK</a>.</p>
  
  <p>Edit: Thinking a little more about it, even if the time is not an issue the situation will roughly be the same. But you will know the performance of the model on new data to a higher certainty with cross validation. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-17T11:14:26.100Z
Answers.Text: What do you mean with: ",it will overfit a more but perform better"?
Answers.UserDisplayName: jennifer ruurs
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-17T11:19:15.640Z
Answers.Text: Using loss as a criteria, I mean both training and test results will be higher, but the ratio of test loss to training loss will be lower compared to 70/30 split.
Answers.UserDisplayName: serali
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-17T11:41:53.360Z
Answers.Text: I meant both training and test results will be lower, not higher in above comment. And also training loss to test loss ratio. It is loss, not accuracy so the lower the better. I am unable to edit the comment, sorry for the second comment.
Answers.UserDisplayName: serali
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-17T11:44:40.430Z
Answers.Text: No i understood what you are trying to say, thank you
Answers.UserDisplayName: jennifer ruurs
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-17T10:49:32.847Z
Answers.LastActivityDate: 2019-11-17T13:35:55.097Z
Answers.LastEditDate: 2019-11-17T13:35:55.097Z
Answers.UserDisplayName: serali
Answers.Body: <p>Both methods are fine if used properly. As a rule of thumb, when training time is not an issue, use split method if you have more data than you can use in your model and cross-validation if not. I would suggest handling overfitting by some other means.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-17T15:35:30.850Z
Answers.LastActivityDate: 2019-11-17T15:35:30.850Z
Answers.UserDisplayName: tjk
Answers.Body: <p>I think this decision will be different from case to case.
  For example, when deep learning networks are used in new architectures such as using a pre-trained network to extract features and using machine learning classifiers to classify data.
  You can no longer use the fold method here!! Because the network is no longer fine-tuning!
  In the 15-15-70 method, we can use more number of runs to reduce the variance.
  In this way, the problem can be solved according to the requested example.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-11T02:04:31.977Z
Answers.LastActivityDate: 2023-02-11T02:04:31.977Z
Answers.UserDisplayName: Ali Achak
Answers.Body: <p>K-fold cross-validation is probably preferred in terms of completeness and generalization: you ensure that the system has seen the complete dataset for training. However, in deep learning this is often not feasible due to time and power constraints. They can both be used, and there is not one <em>better</em> than the other. It really depends on the specific case, the size of the dataset and the time and hardware available. Note that overfitting can be (partially) remedied by things such as dropout.</p>
  
  <p>To be fair: it is fine to have a discussion about this with your colleagues, but as so often there is no one correct answer. If you <em>really</em> want proof, you can test it out and compare them. But performance-wise (i.e. the model's predictive power), the difference will be small.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-17T18:52:04.453Z
Answers.LastActivityDate: 2019-11-18T13:32:52.340Z
Answers.LastEditDate: 2019-11-18T13:32:52.340Z
Answers.UserDisplayName: Bram Vanroy
Body: <p>I want to prevent my model from overfitting. I think that k-fold cross-validation (because it is doing this each time with different datasets) may be more effective than splitting the dataset into training and test datasets to prevent overfitting, but a colleague (who has little experience in ML) says that, to prevent overfitting, the 70/30% split performs better than the k-fold cross-validation. In my opinion, k-fold cross-validation provides a reliable method to test the model performance.</p>
  <p>Is k-fold cross-validation more effective than splitting the dataset into training and test datasets to prevent overfitting? I am not concerned with computational resources.</p>
  
Tags: <machine-learning><comparison><overfitting><cross-validation><k-fold-cv>
Title: Is k-fold cross-validation more effective than splitting the dataset into training and test datasets to prevent overfitting?
UserDisplayName: jennifer ruurs
ThreadText: # Is k-fold cross-validation more effective than splitting the dataset into training and test datasets to prevent overfitting?
  
  (Asked by: username_0 on 2019-11-17)
  
  I want to prevent my model from overfitting. I think that k-fold cross-validation (because it is doing this each time with different datasets) may be more effective than splitting the dataset into training and test datasets to prevent overfitting, but a colleague (who has little experience in ML) says that, to prevent overfitting, the 70/30% split performs better than the k-fold cross-validation. In my opinion, k-fold cross-validation provides a reliable method to test the model performance.
  
  Is k-fold cross-validation more effective than splitting the dataset into training and test datasets to prevent overfitting? I am not concerned with computational resources.
  
  **Tags:** machine-learning, comparison, overfitting, cross-validation, k-fold-cv
  
  ---
  
  ## 4 answers
  
  ### Answer by username_4 on 2019-11-17. Score: 5 | Accepted answer
  
  K-fold cross-validation is probably preferred in terms of completeness and generalization: you ensure that the system has seen the complete dataset for training. However, in deep learning this is often not feasible due to time and power constraints. They can both be used, and there is not one *better* than the other. It really depends on the specific case, the size of the dataset and the time and hardware available. Note that overfitting can be (partially) remedied by things such as dropout.
  
  To be fair: it is fine to have a discussion about this with your colleagues, but as so often there is no one correct answer. If you *really* want proof, you can test it out and compare them. But performance-wise (i.e. the model's predictive power), the difference will be small.
  
  
  ---
  
  ### Answer by username_1 on 2019-11-17. Score: 3
  
  **Purely in terms of overfitting**, and assuming you train both for equal amounts of time, 70/30 is probably better but performance is not going to be very good. Not training on %30 of data will make both training and test results equally bad (in my opinion). But it won't overfit, that is for sure. Cross validation (you have in mind 90/10, I assume) will take a long time, so that won't have enough time to train and it might be overfitting more compared to 70/30, but as it is going to see all training samples %90 at a time, there is a good chance it will train better. So, at the end of the day, it will overfit more but perform better.
  
  If you are asking which is better overall, performance and overfitting, I say it depends on the size of your dataset. If you have millions of samples in it, you can even use a 98/1/1 for training, testing and validation and [still be OK](https://www.coursera.org/learn/deep-neural-network/lecture/cxG1s/train-dev-test-sets).
  
  Edit: Thinking a little more about it, even if the time is not an issue the situation will roughly be the same. But you will know the performance of the model on new data to a higher certainty with cross validation.
  
  > **Comments:**
  > **username_0**: What do you mean with: ",it will overfit a more but perform better"?  
  > **username_1**: Using loss as a criteria, I mean both training and test results will be higher, but the ratio of test loss to training loss will be lower compared to 70/30 split.  
  > **username_1**: I meant both training and test results will be lower, not higher in above comment. And also training loss to test loss ratio. It is loss, not accuracy so the lower the better. I am unable to edit the comment, sorry for the second comment.  
  > **username_0**: No i understood what you are trying to say, thank you  
  
  ---
  
  ### Answer by username_2 on 2019-11-17. Score: 0
  
  Both methods are fine if used properly. As a rule of thumb, when training time is not an issue, use split method if you have more data than you can use in your model and cross-validation if not. I would suggest handling overfitting by some other means.
  
  
  ---
  
  ### Answer by username_3 on 2023-02-11. Score: 0
  
  I think this decision will be different from case to case.
  For example, when deep learning networks are used in new architectures such as using a pre-trained network to extract features and using machine learning classifiers to classify data.
  You can no longer use the fold method here!! Because the network is no longer fine-tuning!
  In the 15-15-70 method, we can use more number of runs to reduce the variance.
  In this way, the problem can be solved according to the requested example.
  
  
  ---
  
  