Answers.Body: <p>I assume you intended to write compute the <em>evaluation metric</em> over the validation set in batches; you do not compute <strong>loss</strong> over the validation set!</p>
  <p>That is quite a standard practice in many academic implementations (<em>because, when the validation set is large enough, the memory will be a constraint</em>), however, be sure to take the average of the values over all the batches. Using a K-fold setup will increase the confidence in the reported values.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-15T14:26:25.807Z
Answers.LastActivityDate: 2021-02-15T14:26:25.807Z
Answers.UserDisplayName: anurag
Body: <p>I have about 2000 items in my validation set, would it be reasonable to calculate the loss/error after each epoch on just a subset instead of the whole set, if calculating the whole dataset is very slow?</p>
  <p>Would taking random mini-batches to calculate loss be a good idea as your network wouldn't have a constant set? Should I just shrink the size of my validation set?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-02-15T12:24:39.590Z
Comments.Text: I would also suggest that you explain a little bit the model you're using, how much time it takes to compute the validation loss and what is the task you're trying to solve.
Comments.UserDisplayName: nbro
Tags: <machine-learning><loss><batch-learning><validation-loss>
Title: Is it okay to calculate the validation loss over batches instead of the whole validation set for speed purposes?
UserDisplayName: Ilknur Mustafa
ThreadText: # Is it okay to calculate the validation loss over batches instead of the whole validation set for speed purposes?
  
  (Asked by: username_0 on 2021-02-15)
  
  I have about 2000 items in my validation set, would it be reasonable to calculate the loss/error after each epoch on just a subset instead of the whole set, if calculating the whole dataset is very slow?
  
  Would taking random mini-batches to calculate loss be a good idea as your network wouldn't have a constant set? Should I just shrink the size of my validation set?
  
  **Tags:** machine-learning, loss, batch-learning, validation-loss
  
  > **Comments:**
  > **username_1**: I would also suggest that you explain a little bit the model you're using, how much time it takes to compute the validation loss and what is the task you're trying to solve.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-02-15. Score: 1
  
  I assume you intended to write compute the *evaluation metric* over the validation set in batches; you do not compute **loss** over the validation set!
  
  That is quite a standard practice in many academic implementations (*because, when the validation set is large enough, the memory will be a constraint*), however, be sure to take the average of the values over all the batches. Using a K-fold setup will increase the confidence in the reported values.
  
  
  ---
  
  