The mask tensor is initialized via torch.empty, which uses uninitialized memory and thus random values.
Based on the output of torch.unique(mask) it seems you haven’t set all values in the mask and thus the last value is coming from the uninitialized memory.
Could you check where this index is used and which color can be found there? I guess you might have missed a particular color (or it was created due to e.g. linear interpolation during resizing).