If your target only contains 4 classes, you should be able to map the values to just these 4 class indices [0, 1, 2 ,3].
Does your target contain 4 discrete values in the range [0, 255]? If not, you might need to create bins for the noisy values to map them to the needed class indices.