Answers.Body: <p>Yes! This is crucial.</p>
  <p>If you rotate your input images for segmentation, you need to rotate the output masks as well. Otherwise the loss of your network will not be correctly calculated and your network will not learn how to generalize to rotated input images.</p>
  <p>If you use keras, you can use two <code>ImageDataGenerator</code> classes, one for the images and one for the masks, with the <strong>same</strong> random seed and augmentation parameters. It looks something like this:</p>
  <pre><code>data_gen_args = dict(rotation_range=45)
  
  image_datagen = ImageDataGenerator(**data_gen_args)
  mask_datagen = ImageDataGenerator(**data_gen_args)
  
  image_generator = image_datagen.flow(train,seed=SEED)
  mask_generator = mask_datagen.flow(y_train,seed=SEED)
  train_generator = zip(image_generator, mask_generator)
  
  model.compile(...)
  model.fit_generator(train_generator, ...)
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-10T18:29:24.127Z
Answers.Text: This is great thank you! So what I ended up doing was resizing the masks and concatenating the images with the masks and then 
  tf.keras.preprocessing.images.random_shear((images,masks),*args) and then resizing the masks after. Would setting the same seed be sufficient to make sure the rotations or shearing agreed on both. 
  Lastly, is there anyway to use ImageDataGenerator with a custom generator also?
Answers.UserDisplayName: ADA
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-10T08:08:05.053Z
Answers.LastActivityDate: 2020-07-10T08:08:05.053Z
Answers.UserDisplayName: Tinu
Body: <p>I am training a neural network that takes an input <code>(H, W, 3)</code> and has the output of size <code>(H', W', C)</code>. Now, to augment my dataset, since I only have 45k images, I am using the following in my custom data generator</p>
  <pre><code>def Generator():
  img=cv2.imread(trainDir+'\'+imgpath)
  img=tf.keras.preprocessing.image.random_rotation(img,20m,row_axis=0,col_axis=1,channel_axis=2)
  
  output_mask=np.load(trainDir+'\'+maskpath)
  
  yield(img/255-.5,output_mask)
  </code></pre>
  <p>Since I am rotating my input images and the output_masks are generated from information about the input (specifically, heat maps around the joint locations) do I also need to rotate the masks as well?</p>
  
Tags: <neural-networks><convolutional-neural-networks><data-preprocessing><data-augmentation>
Title: Do I need to rotate the masks, if I also rotate the images and the masks are generated from the input?
UserDisplayName: ADA
ThreadText: # Do I need to rotate the masks, if I also rotate the images and the masks are generated from the input?
  
  (Asked by: username_0 on 2020-07-09)
  
  I am training a neural network that takes an input (H, W, 3) and has the output of size (H', W', C). Now, to augment my dataset, since I only have 45k images, I am using the following in my custom data generator
  def Generator():
  img=cv2.imread(trainDir+'\'+imgpath)
  img=tf.keras.preprocessing.image.random_rotation(img,20m,row_axis=0,col_axis=1,channel_axis=2)
  
  output_mask=np.load(trainDir+'\'+maskpath)
  
  yield(img/255-.5,output_mask)
  
  Since I am rotating my input images and the output_masks are generated from information about the input (specifically, heat maps around the joint locations) do I also need to rotate the masks as well?
  
  **Tags:** neural-networks, convolutional-neural-networks, data-preprocessing, data-augmentation
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-07-10. Score: 1 | Accepted answer
  
  Yes! This is crucial.
  If you rotate your input images for segmentation, you need to rotate the output masks as well. Otherwise the loss of your network will not be correctly calculated and your network will not learn how to generalize to rotated input images.
  If you use keras, you can use two ImageDataGenerator classes, one for the images and one for the masks, with the same random seed and augmentation parameters. It looks something like this:
  data_gen_args = dict(rotation_range=45)
  
  image_datagen = ImageDataGenerator(**data_gen_args)
  mask_datagen = ImageDataGenerator(**data_gen_args)
  
  image_generator = image_datagen.flow(train,seed=SEED)
  mask_generator = mask_datagen.flow(y_train,seed=SEED)
  train_generator = zip(image_generator, mask_generator)
  
  model.compile(...)
  model.fit_generator(train_generator, ...)
  
  > **Comments:**
  > **username_0**: This is great thank you! So what I ended up doing was resizing the masks and concatenating the images with the masks and then  tf.keras.preprocessing.images.random_shear((images,masks),*args) and then resizing the masks after. Would setting the same seed be sufficient to make sure the rotations or shearing agreed on both.  Lastly, is there anyway to use ImageDataGenerator with a custom generator also?  
  
  ---
  
  