Taking a closer look, I see that mask is initialized with torch.empty. You might want to do torch.zeros to prevent any unlabeled pixels from looking strange unless you are certain every single pixel will be labeled.
I would then check that mask makes sense for each of the individual classes (e.g., can you try viewing the mask with just the first class, just the second, and so on?) to make sure that nothing strange like different classes overwriting each other is happening.
As a sanity check, could you print the shapes of target, idx,  and validx?