the randomness could be due to dataset shuffling. I recommend iterating through the dataset (without training, just iterating to make it fast) and print the image names and check if it always happens on the same images. Then, manually open these images to see if there’s an issue with these.