Answers.Body: <p>Data that has a size of a factor of <span class="math-container">$2$</span> (aka <span class="math-container">$2^n$</span> for some integer <span class="math-container">$n$</span>) allows for easier memory management because data can be organized contiguously (without gaps). This allows for faster memory reading and thus faster iteration time in general. From a computational point of view, this is important as it can be taken advantage of by the compiler and speed up iteration loops. This is why batch sizes are chosen as such in practice. However, this does not necessarily imply better training results.</p>
  <p>In regards to your question of &quot;Can I use other values for batch size?&quot;:</p>
  <p>Yes, you can use different values, and, for the most part, you probably won't see the difference in computational performance because of the speed of modern training APIs. So, until training a large model with a lot of compute where this optimization has a larger impact, feel free to experiment :)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-30T15:32:18.983Z
Answers.LastActivityDate: 2020-10-30T18:59:30.187Z
Answers.LastEditDate: 2020-10-30T18:59:30.187Z
Answers.UserDisplayName: Jaden Travnik
Body: <p>I am training a CNN with a batch size of 128, but I have some fluctuations in the validation loss, which are greater than one. I want to increase my batch size to 150 or 200, but, in the code examples I have come across, the batch size is always something like 32, 64, 128, or 256. Is it a rule? Can I use other values for it?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-30T09:57:12.897Z
Comments.Text: related: https://stats.stackexchange.com/questions/153531/what-is-batch-size-in-neural-network
Comments.UserDisplayName: pasaba por aqui
Tags: <machine-learning><convolutional-neural-networks><training><batch-size>
Title: Are there any rules for choosing batch size?
UserDisplayName: SahaTib
ThreadText: # Are there any rules for choosing batch size?
  
  (Asked by: username_0 on 2020-10-29)
  
  I am training a CNN with a batch size of 128, but I have some fluctuations in the validation loss, which are greater than one. I want to increase my batch size to 150 or 200, but, in the code examples I have come across, the batch size is always something like 32, 64, 128, or 256. Is it a rule? Can I use other values for it?
  
  **Tags:** machine-learning, convolutional-neural-networks, training, batch-size
  > **Comments:**
  > **username_1**: related: https://stats.stackexchange.com/questions/153531/what-is-batch-size-in-neural-network  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-10-30. Score: 2
  
  Data that has a size of a factor of $2$ (aka $2^n$ for some integer $n$) allows for easier memory management because data can be organized contiguously (without gaps). This allows for faster memory reading and thus faster iteration time in general. From a computational point of view, this is important as it can be taken advantage of by the compiler and speed up iteration loops. This is why batch sizes are chosen as such in practice. However, this does not necessarily imply better training results.
  
  In regards to your question of "Can I use other values for batch size?":
  
  Yes, you can use different values, and, for the most part, you probably won't see the difference in computational performance because of the speed of modern training APIs. So, until training a large model with a lot of compute where this optimization has a larger impact, feel free to experiment :)
  
  
  ---
  
  