<p>Yes! This is crucial.</p>
<p>If you rotate your input images for segmentation, you need to rotate the output masks as well. Otherwise the loss of your network will not be correctly calculated and your network will not learn how to generalize to rotated input images.</p>
<p>If you use keras, you can use two <code>ImageDataGenerator</code> classes, one for the images and one for the masks, with the <strong>same</strong> random seed and augmentation parameters. It looks something like this:</p>
<pre><code>data_gen_args = dict(rotation_range=45)

image_datagen = ImageDataGenerator(**data_gen_args)
mask_datagen = ImageDataGenerator(**data_gen_args)

image_generator = image_datagen.flow(train,seed=SEED)
mask_generator = mask_datagen.flow(y_train,seed=SEED)
train_generator = zip(image_generator, mask_generator)

model.compile(...)
model.fit_generator(train_generator, ...)
</code></pre>

CC BY-SA 4.0
2020-07-10T18:29:24.127Z
This is great thank you! So what I ended up doing was resizing the masks and concatenating the images with the masks and then 
tf.keras.preprocessing.images.random_shear((images,masks),*args) and then resizing the masks after. Would setting the same seed be sufficient to make sure the rotations or shearing agreed on both. 
Lastly, is there anyway to use ImageDataGenerator with a custom generator also?
ADA
CC BY-SA 4.0
2020-07-10T08:08:05.053Z
2020-07-10T08:08:05.053Z
Tinu
<p>I am training a neural network that takes an input <code>(H, W, 3)</code> and has the output of size <code>(H', W', C)</code>. Now, to augment my dataset, since I only have 45k images, I am using the following in my custom data generator</p>
<pre><code>def Generator():
img=cv2.imread(trainDir+'\'+imgpath)
img=tf.keras.preprocessing.image.random_rotation(img,20m,row_axis=0,col_axis=1,channel_axis=2)

output_mask=np.load(trainDir+'\'+maskpath)

yield(img/255-.5,output_mask)
</code></pre>
<p>Since I am rotating my input images and the output_masks are generated from information about the input (specifically, heat maps around the joint locations) do I also need to rotate the masks as well?</p>

<neural-networks><convolutional-neural-networks><data-preprocessing><data-augmentation>
Do I need to rotate the masks, if I also rotate the images and the masks are generated from the input?
ADA
# Do I need to rotate the masks, if I also rotate the images and the masks are generated from the input?

(Asked by: username_0 on 2020-07-09)

I am training a neural network that takes an input (H, W, 3) and has the output of size (H', W', C). Now, to augment my dataset, since I only have 45k images, I am using the following in my custom data generator
def Generator():
img=cv2.imread(trainDir+'\'+imgpath)
img=tf.keras.preprocessing.image.random_rotation(img,20m,row_axis=0,col_axis=1,channel_axis=2)

output_mask=np.load(trainDir+'\'+maskpath)

yield(img/255-.5,output_mask)

Since I am rotating my input images and the output_masks are generated from information about the input (specifically, heat maps around the joint locations) do I also need to rotate the masks as well?

**Tags:** neural-networks, convolutional-neural-networks, data-preprocessing, data-augmentation


---

## 1 answers

### Answer by username_1 on 2020-07-10. Score: 1 | Accepted answer

Yes! This is crucial.
If you rotate your input images for segmentation, you need to rotate the output masks as well. Otherwise the loss of your network will not be correctly calculated and your network will not learn how to generalize to rotated input images.
If you use keras, you can use two ImageDataGenerator classes, one for the images and one for the masks, with the same random seed and augmentation parameters. It looks something like this:
data_gen_args = dict(rotation_range=45)

image_datagen = ImageDataGenerator(**data_gen_args)
mask_datagen = ImageDataGenerator(**data_gen_args)

image_generator = image_datagen.flow(train,seed=SEED)
mask_generator = mask_datagen.flow(y_train,seed=SEED)
train_generator = zip(image_generator, mask_generator)

model.compile(...)
model.fit_generator(train_generator, ...)

> **Comments:**
> **username_0**: This is great thank you! So what I ended up doing was resizing the masks and concatenating the images with the masks and then  tf.keras.preprocessing.images.random_shear((images,masks),*args) and then resizing the masks after. Would setting the same seed be sufficient to make sure the rotations or shearing agreed on both.  Lastly, is there anyway to use ImageDataGenerator with a custom generator also?  

---

