While the class5 channel would have probabilities (or logits) for this class, the max. logit for these pixels could still be class0. Did you compare the values of both classes for the area of interest?