The output points towards an uninitialized tensor and looks like a bug.
I’m not able to reproduce it using this code snippet:
for _ in range(10000):
    N, H, W = 8, 24, 24
    gt_arr = np.random.randint(0, 3, (N, H, W))
    gt_arr_hot = nn.functional.one_hot(torch.from_numpy(gt_arr).to(torch.int64), num_classes=3).permute(0,3,1,2)
    
    if gt_arr_hot.unique().nelement() != 2:
        print('Error')
        print(np.unique(gt_arr))
        print(gt_arr_hot.unique())

Could you run it and check, if you are able to get these invalid outputs using it?