Yes, when I print the shape of image after converting it into numpy array, even reading from both cv2 and PIL, the output shape is 3*512*512 which I corrected using np.transpose(imgs,(2,0,1)).
I got your point regarding the mapping but how to define the mapping for RGB images, my masks have 4 classes- Blue,Red,Green and black background.
I am not resizing the mask images but I am Random Cropping the original image to a smaller size and my final output segmented image is same size as of cropped input image.Is it the right way to do it, are there any performance issues with it?