Answers.Body: <blockquote>
    <p>I am not familiar with using batches during network evaluation. Can someone explain what is the reason behind using it and what are advantages and disadvantages?</p>
  </blockquote>
  
  <p>It is usually just for memory use limitation vs speed of assessment. Larger batches evaluate faster on parallelised systems such as GPUs, but use more memory to process. Test results should be identical, with same size of dataset and same model, regardless of batch size.</p>
  
  <p>Typically you would set batch size at least high enough to take advantage of available hardware, and after that as high as you dare without taking the risk of getting memory errors. Generally there is less to gain than with training optimisation though, so it is not worth spending a huge amount of time optimising the batch size to each model you want to test. In most code I have seen, users pick a moderate "safe" value that will speed up testing but doesn't risk failing if you wanted to add a few layers to the model and check what that does.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-02-26T07:37:24.623Z
Answers.LastActivityDate: 2019-02-26T07:37:24.623Z
Answers.UserDisplayName: Neil Slater
Body: <p>If one examines the <em>SSD: Single Shot MultiBox Detector</em> code from <a href="https://github.com/weiliu89/caffe/tree/ssd" rel="nofollow noreferrer">this GitHub repository</a>, it can be seen that, for a testing phase (evaluating network on test data set), there is a parameter <code>test batch size</code>. It is not mentioned in the paper.</p>
  <p>I am not familiar with using batches during network evaluation. Can someone explain what is the reason behind using it and what are advantages and disadvantages?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-02-26T14:38:34.457Z
Comments.Text: Actually, I've asked a similar question before in StackOverflow: [Why does prediction needs batch size in Keras?](https://stackoverflow.com/q/37911321/2147347).
Comments.UserDisplayName: malioboro
Tags: <deep-learning><testing><batch-size><test-datasets><single-shot-multibox-detector>
Title: What is the reason behind using a test batch size?
UserDisplayName: carobnodrvo
ThreadText: # What is the reason behind using a test batch size?
  
  (Asked by: username_0 on 2019-01-25)
  
  If one examines the *SSD: Single Shot MultiBox Detector* code from [this GitHub repository](https://github.com/weiliu89/caffe/tree/ssd), it can be seen that, for a testing phase (evaluating network on test data set), there is a parameter `test batch size`. It is not mentioned in the paper.
  
  I am not familiar with using batches during network evaluation. Can someone explain what is the reason behind using it and what are advantages and disadvantages?
  
  **Tags:** deep-learning, testing, batch-size, test-datasets, single-shot-multibox-detector
  > **Comments:**
  > **username_1**: Actually, I've asked a similar question before in StackOverflow: [Why does prediction needs batch size in Keras?](https://stackoverflow.com/q/37911321/2147347).  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2019-02-26. Score: 2 | Accepted answer
  
  > I am not familiar with using batches during network evaluation. Can someone explain what is the reason behind using it and what are advantages and disadvantages?
  
  It is usually just for memory use limitation vs speed of assessment. Larger batches evaluate faster on parallelised systems such as GPUs, but use more memory to process. Test results should be identical, with same size of dataset and same model, regardless of batch size.
  
  Typically you would set batch size at least high enough to take advantage of available hardware, and after that as high as you dare without taking the risk of getting memory errors. Generally there is less to gain than with training optimisation though, so it is not worth spending a huge amount of time optimising the batch size to each model you want to test. In most code I have seen, users pick a moderate "safe" value that will speed up testing but doesn't risk failing if you wanted to add a few layers to the model and check what that does.
  
  
  ---
  
  