Answers.Body: <p>I guess you could train your model with 10 different folds and in each fold calculate the average accuracy. So you would have 10 values - one corresponding to each fold. And then take the mean of all of them to get the average accuracy of your model.</p>
  
  <p>Your first option doesn't seem great because you take the highest accuracy among folds. If for some reason, the variance between accuracies is high for a fold, this would bias your numbers. Taking mean or maybe median of accuracies might be more reasonable.</p>
  
  <p>Does that help?</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-13T00:31:33.197Z
Answers.Text: But then how should I calculate the average accuracy of each fold?
Answers.UserDisplayName: Juna
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-14T04:42:13.953Z
Answers.Text: Unless your model has some dynamic component, maybe like RF, I don't think you would need to average for each fold. Actually, you can just evaluate your model once over the model and take that accuracy as the accuracy for the fold.
Answers.UserDisplayName: pecey
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-12T21:13:07.043Z
Answers.LastActivityDate: 2020-05-12T21:13:07.043Z
Answers.UserDisplayName: pecey
Answers.Body: <p>In most cases we choose to take the mean of the k accuracies of k-fold cross validation; that is each time take the one that corresponds to the fold and when every fold has been used as validation set, find the mean accuracy of them.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-12T21:50:44.043Z
Answers.LastActivityDate: 2020-05-12T21:50:44.043Z
Answers.UserDisplayName: ddaedalus
Body: <p>I'm struggling with calculating accuracy
  when I do cross-validation for a deep learning model.
  I have two candidates for doing this.
  1. Train a model with 10 different folds and get the best accuracy of them(so I get 10 best accuracies) and average them.
  2. Train a model with 10 different folds and get 10 accuracy learning curves. Now, average these learning curves by calculating the mean of 10 accuracies of each epoch. So now we get one averaged accuracy learning curve and find the highest accuracy from this curve.</p>
  
  <p>Among these two candidates which one is correct??</p>
  
Tags: <deep-learning><cross-validation>
Title: Calculating accuracy for cross validation
UserDisplayName: Juna
ThreadText: # Calculating accuracy for cross validation
  
  (Asked by: username_0 on 2020-05-12)
  
  I'm struggling with calculating accuracy
  when I do cross-validation for a deep learning model.
  I have two candidates for doing this.
  1. Train a model with 10 different folds and get the best accuracy of them(so I get 10 best accuracies) and average them.
  2. Train a model with 10 different folds and get 10 accuracy learning curves. Now, average these learning curves by calculating the mean of 10 accuracies of each epoch. So now we get one averaged accuracy learning curve and find the highest accuracy from this curve.
  
  Among these two candidates which one is correct??
  
  **Tags:** deep-learning, cross-validation
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2020-05-12. Score: 2
  
  I guess you could train your model with 10 different folds and in each fold calculate the average accuracy. So you would have 10 values - one corresponding to each fold. And then take the mean of all of them to get the average accuracy of your model.
  
  Your first option doesn't seem great because you take the highest accuracy among folds. If for some reason, the variance between accuracies is high for a fold, this would bias your numbers. Taking mean or maybe median of accuracies might be more reasonable.
  
  Does that help?
  
  > **Comments:**
  > **username_0**: But then how should I calculate the average accuracy of each fold?  
  > **username_1**: Unless your model has some dynamic component, maybe like RF, I don't think you would need to average for each fold. Actually, you can just evaluate your model once over the model and take that accuracy as the accuracy for the fold.  
  
  ---
  
  ### Answer by username_2 on 2020-05-12. Score: 0
  
  In most cases we choose to take the mean of the k accuracies of k-fold cross validation; that is each time take the one that corresponds to the fold and when every fold has been used as validation set, find the mean accuracy of them.
  
  
  ---
  
  