Answers.Body: <p><strong>tl;dr</strong> subtracting the mean and dividing by the standard deviation is theoretically more sound, but is impractical compared to dividing by <span class="math-container">$255$</span>.</p>
  <hr />
  <h3>Explanation</h3>
  <p>As you know neural networks perform better when their input is scaled. The 2 most common ways to perform scaling are:</p>
  <ul>
  <li><strong>normalization</strong>, where you want to scale the image to the <span class="math-container">$[0, 1]$</span> range.</li>
  <li><strong>standardization</strong>, where you want to bring the mean to <span class="math-container">$0$</span> and the standard deviation to <span class="math-container">$1$</span>.</li>
  </ul>
  <p>Theoretically, I think, the latter has some advantages, however in practice there is not a significant difference, especially if the network has normalization layers.</p>
  <p>Images are usually stored in 8-bit color mode, meaning that they take integer values from 0 to 255. Because you know the minimum, and maximum values that each pixel can take, it is extremely <strong>easy</strong> to normalize an image (i.e. you simply have to divide each pixel value by <span class="math-container">$255$</span>).</p>
  <p>On the other hand it is much <strong>less practical</strong> to compute the mean and standard deviation of the pixel intensities of the training set. Keep in mind that in most cases the training set doesn't even fit into memory, and you have to approximate the true mean/std by moving averages (i.e. load a few images, compute their mean/std, close them, load some more, add their mean to the MA, ...). But why bother if you can similar results by simply dividing each image by <span class="math-container">$255$</span>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-01T09:47:18.837Z
Answers.Text: I guess you mean Batch Nornalization by normalization layers right?
Answers.UserDisplayName: Baran Aldemir
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-01T19:40:31.117Z
Answers.Text: Yes, batch norm, layer norm, group norm, instance norm, etc.
Answers.UserDisplayName: Djib2011
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-01T08:59:26.870Z
Answers.LastActivityDate: 2021-03-01T08:59:26.870Z
Answers.UserDisplayName: Djib2011
Body: <p>I have seen people normalize images by just dividing 255. But why? Why not use mean normalization or Z-score Normalization? I also came across <a href="https://stackoverflow.com/questions/55859716/does-normalizing-images-by-dividing-by-255-leak-information-between-train-and-te">this StackOverflow topic</a> while searching but the answers there were not enough enlightening for me.</p>
  
Tags: <convolutional-neural-networks><data-preprocessing><normalisation><standardisation>
Title: How to normalize images before training?
UserDisplayName: Baran Aldemir
ThreadText: # How to normalize images before training?
  
  (Asked by: username_0 on 2021-02-28)
  
  I have seen people normalize images by just dividing 255. But why? Why not use mean normalization or Z-score Normalization? I also came across [this StackOverflow topic](https://stackoverflow.com/questions/55859716/does-normalizing-images-by-dividing-by-255-leak-information-between-train-and-te) while searching but the answers there were not enough enlightening for me.
  
  **Tags:** convolutional-neural-networks, data-preprocessing, normalisation, standardisation
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-03-01. Score: 1
  
  **tl;dr** subtracting the mean and dividing by the standard deviation is theoretically more sound, but is impractical compared to dividing by $255$.
  
  ---
  
  ### Explanation
  
  As you know neural networks perform better when their input is scaled. The 2 most common ways to perform scaling are:
  
  * **normalization**, where you want to scale the image to the $[0, 1]$ range.
  * **standardization**, where you want to bring the mean to $0$ and the standard deviation to $1$.
  
  Theoretically, I think, the latter has some advantages, however in practice there is not a significant difference, especially if the network has normalization layers.
  
  Images are usually stored in 8-bit color mode, meaning that they take integer values from 0 to 255. Because you know the minimum, and maximum values that each pixel can take, it is extremely **easy** to normalize an image (i.e. you simply have to divide each pixel value by $255$).
  
  On the other hand it is much **less practical** to compute the mean and standard deviation of the pixel intensities of the training set. Keep in mind that in most cases the training set doesn't even fit into memory, and you have to approximate the true mean/std by moving averages (i.e. load a few images, compute their mean/std, close them, load some more, add their mean to the MA, ...). But why bother if you can similar results by simply dividing each image by $255$.
  
  > **Comments:**
  > **username_0**: I guess you mean Batch Nornalization by normalization layers right?  
  > **username_1**: Yes, batch norm, layer norm, group norm, instance norm, etc.  
  
  ---
  
  