I assume by “classifies some of the classes” you mean that your predicted segmentations ignore some classes completely?
If your target classes are imbalanced, you could try to weight the pixel values according to the number of pixels for the particular class.