Thanks for the image.
The target image still has 20 different target values.
Using a histogram and a quick visual inspection of the array, it seems that the only valid values should be:

0 for background
15 for class0
40 for class1

All other values seem to be created by some interpolation technique.
Did you resampled/interpolated the target image using something like bilinear interpolation?
If so, you should use the nearest interpolation technique to avoid undefined classes.
However, you could just remove the interp values and go with the images you have now.
Try the following code to clean your targets:
im_arr = np.array(label)
# Remove interp values
im_arr[(im_arr!=0) & (im_arr!=15) & (im_arr!=40)] = 0
# Set to classes
im_arr[im_arr==15] = 1
im_arr[im_arr==40] = 2
print(np.unique(im_arr))

Using this code you will get small borders, where the interpolation was previously.
If you have the raw data, you could re-create the dataset with the nearest interpolation, if not, this could be good enough.
Also, change the number of classes to 3, since you have a background class and two plant classes.