That is an interesting insight. Is your thought that this mixup is only happening in my training loop, so that the labels are mixed up but fixed and therefore learnable (explaining the improvement in loss during training)? And then when I am no longer in the training loop, I evaluate the model in a way where the labels are not mixed up, which is why the results don’t hold up?