alex_d:

Like in my masked images, all the background pixels are black, and the ones that belong to the car class are blue. Is using that enough to map the classes correctly?


Most likely not. Base on your description, it seems you are dealing with color images as masks.
For a usual segmentation use case, the mask should be a LongTensor with the shape [batch_size, height, width] containing class indices.
Have a look at this post 22 to see, how a mapping between the color values and your class indices can be created.