Answers.Body: <p>So far, it seems this is more a software &quot;integration&quot; issue. One great tip from <a href="http://karpathy.github.io/2019/04/25/recipe/" rel="nofollow noreferrer">http://karpathy.github.io/2019/04/25/recipe/</a> is to visualize everything as often as you can during development. For data augmentation, try to visualize the image right before it enters your convnet.</p>
  <p>What I found is a bug can happen if your particular image transform library try to detect if you have rescaled or not by just checking the data type. i.e. if it sees float, it will assume [0, 1] or if an integer, it will assume [0, 255].</p>
  <p>This can end badly if you resize (without rescaling at the same time) using tf.image.resize(...). Resizing an image (e.g. to 224x224x3) has the &quot;side effect&quot; of converting the pixel value to a float, while still having an (approx.) range of 0.0-255.0. This is such that a downstream library will see a float and make the wrong assumption of [0, 1] range and totally mess up the transform. If you use any color related transforms, definitely be careful, they are usually dependent on pixel value range assumption.</p>
  <p>Because of this, if you use tf.image, you may find it safer to do resize and rescale at the same time to ensure that float =&gt; [0, 1] such that your downstream color augmentation library may most likely work. Or you just have to manage the type carefully and do tf.cast(...) wherever necessary to satisfy any library assumption.</p>
  <p>At the end, a human eye will still have to verify them.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-05-02T20:12:40.597Z
Answers.LastActivityDate: 2021-05-02T20:12:40.597Z
Answers.UserDisplayName: kawingkelvin
Body: <p>During image preprocessing pipeline, should one rescale each pixel value to [0, 1] by dividing 255 first, and then perform data transformation such as color distortion, gaussian blur? or vice versa?</p>
  <p>I believe for correctness, it may depend on the particular image transformation algorithm, or if you use some libraries. Is there any general advice? If anyone has experience trying both before or after, please share, particularly if you use external library or framework that may make implicit assumption to the value range of the pixel.</p>
  
Tags: <convolutional-neural-networks><image-processing><data-augmentation>
Title: Should one rescale (normalize) image before or after data augmentation?
UserDisplayName: kawingkelvin
ThreadText: # Should one rescale (normalize) image before or after data augmentation?
  
  (Asked by: username_0 on 2021-05-02)
  
  During image preprocessing pipeline, should one rescale each pixel value to [0, 1] by dividing 255 first, and then perform data transformation such as color distortion, gaussian blur? or vice versa?
  
  I believe for correctness, it may depend on the particular image transformation algorithm, or if you use some libraries. Is there any general advice? If anyone has experience trying both before or after, please share, particularly if you use external library or framework that may make implicit assumption to the value range of the pixel.
  
  **Tags:** convolutional-neural-networks, image-processing, data-augmentation
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2021-05-02. Score: 1
  
  So far, it seems this is more a software "integration" issue. One great tip from <http://karpathy.github.io/2019/04/25/recipe/> is to visualize everything as often as you can during development. For data augmentation, try to visualize the image right before it enters your convnet.
  
  What I found is a bug can happen if your particular image transform library try to detect if you have rescaled or not by just checking the data type. i.e. if it sees float, it will assume [0, 1] or if an integer, it will assume [0, 255].
  
  This can end badly if you resize (without rescaling at the same time) using tf.image.resize(...). Resizing an image (e.g. to 224x224x3) has the "side effect" of converting the pixel value to a float, while still having an (approx.) range of 0.0-255.0. This is such that a downstream library will see a float and make the wrong assumption of [0, 1] range and totally mess up the transform. If you use any color related transforms, definitely be careful, they are usually dependent on pixel value range assumption.
  
  Because of this, if you use tf.image, you may find it safer to do resize and rescale at the same time to ensure that float => [0, 1] such that your downstream color augmentation library may most likely work. Or you just have to manage the type carefully and do tf.cast(...) wherever necessary to satisfy any library assumption.
  
  At the end, a human eye will still have to verify them.
  
  
  ---
  
  