Answers.Body: <p><a href="https://stats.stackexchange.com/a/153535/264183">This</a> Cross Validated post might answer your question.</p>
  <p>In a nutshell:</p>
  <ul>
  <li><p>A single batch (that is all your data in one batch) will result in a smooth trajectory on the loss surface. The drawback is that all your data might not fit into your memory. Which is highly likely for ~100k images.</p>
  </li>
  <li><p>One image per batch (batch size = no. examples) will result in a more stochastic trajectory since the gradients are calculated on a single example. Advantages are of computational nature and faster training time.</p>
  </li>
  </ul>
  <p>The middle way is to choose the batch size in such a way that your batch fits into memory and gradients behave less 'noisy'. To be honest there is no 'golden' number, personally I like to choose powers of two.</p>
  <p>Don't worry that your data is not divisible by the batch size. Libraries will take care about that internally, the last batch will just be a smaller than the defined batch size (<span class="math-container">$N \text{ mod } b$</span>).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-19T07:41:49.417Z
Answers.LastActivityDate: 2020-08-19T07:41:49.417Z
Answers.UserDisplayName: Tinu
Answers.Body: <p>From Andrew lesson on Coursera, <code>batch_size</code> should be the power of 2, ex: 512, 1024, 2048. It will faster for training.</p>
  <p>And you don't need to drop your last images to <code>batch_size</code> of 5 for example. The library likes Tensorflow or Pytorch, the last <code>batch_size</code> will be <code>number_training_images % 5</code> which 5 is your <code>batch_size</code>.</p>
  <p>Last but not least, batch_size need to fit your memory training (CPU or GPU). You can try several large batch_size to know which value is not out of memory. The smaller <code>number_mini_batch = number_training_image//batch_size + 1</code>, the faster for training time.</p>
  <p>Hope they can help you!</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-20T18:04:10.807Z
Answers.LastActivityDate: 2020-08-20T18:04:10.807Z
Answers.UserDisplayName: Truong Hoang
Body: <p>I have an image dataset, which is composed of 113695 images for training and 28424 images for validation. Now, when I use <code>ImageDataGenerator</code> and <code>flow_from_dataframe</code>, it as the parameter <code>batch_size</code>.</p>
  <p>How can I take the correct number for <code>batch_size</code> because both numbers cannot be divided by the same number? Should I need to drop four images in the validation data to make them <code>batch_size</code> of 5? Or is there another way?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-08-21T11:15:54.917Z
Comments.Text: Note that programming issues are off-topic here. I am not completely sure whether this is only a programming issue or not, so I will leave it open for now. Can you clarify if this is a programming issue or not? Please, read our on-topic page: https://ai.stackexchange.com/help/on-topic.
Comments.UserDisplayName: nbro
Tags: <deep-learning><computer-vision><tensorflow><data-preprocessing>
Title: How to take the optimal batch_size for training a model?
UserDisplayName: Ravi Teja
ThreadText: # How to take the optimal batch_size for training a model?
  
  (Asked by: username_0 on 2020-08-19)
  
  I have an image dataset, which is composed of 113695 images for training and 28424 images for validation. Now, when I use `ImageDataGenerator` and `flow_from_dataframe`, it as the parameter `batch_size`.
  
  How can I take the correct number for `batch_size` because both numbers cannot be divided by the same number? Should I need to drop four images in the validation data to make them `batch_size` of 5? Or is there another way?
  
  **Tags:** deep-learning, computer-vision, tensorflow, data-preprocessing
  > **Comments:**
  > **username_1**: Note that programming issues are off-topic here. I am not completely sure whether this is only a programming issue or not, so I will leave it open for now. Can you clarify if this is a programming issue or not? Please, read our on-topic page: https://ai.stackexchange.com/help/on-topic.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2020-08-19. Score: 1
  
  [This](https://stats.stackexchange.com/a/153535/264183) Cross Validated post might answer your question.
  
  In a nutshell:
  
  * A single batch (that is all your data in one batch) will result in a smooth trajectory on the loss surface. The drawback is that all your data might not fit into your memory. Which is highly likely for ~100k images.
  * One image per batch (batch size = no. examples) will result in a more stochastic trajectory since the gradients are calculated on a single example. Advantages are of computational nature and faster training time.
  
  The middle way is to choose the batch size in such a way that your batch fits into memory and gradients behave less 'noisy'. To be honest there is no 'golden' number, personally I like to choose powers of two.
  
  Don't worry that your data is not divisible by the batch size. Libraries will take care about that internally, the last batch will just be a smaller than the defined batch size ($N \text{ mod } b$).
  
  
  ---
  
  ### Answer by username_3 on 2020-08-20. Score: 0
  
  From Andrew lesson on Coursera, `batch_size` should be the power of 2, ex: 512, 1024, 2048. It will faster for training.
  
  And you don't need to drop your last images to `batch_size` of 5 for example. The library likes Tensorflow or Pytorch, the last `batch_size` will be `number_training_images % 5` which 5 is your `batch_size`.
  
  Last but not least, batch\_size need to fit your memory training (CPU or GPU). You can try several large batch\_size to know which value is not out of memory. The smaller `number_mini_batch = number_training_image//batch_size + 1`, the faster for training time.
  
  Hope they can help you!
  
  
  ---
  
  