I found the issue. It was a faulty dataset.
My aim here is to have an understanding as to how the segmentation code works in the torchvision library. Now, there are 23 classes (0 being the unlabeled). So, I put ignore_index=0 and num_classess=23
The original images are 4000x6000. The network sees random crops of 480x480. The mask is a single channel of target class values (0-22). Obviously, some of the targets, because of cropping, do not have max value as 22.
Now, I get error : target 22 is out of bounds
What am I missing?