All the images are uniform (taken from the same camera in the same conditions). I have a custom dataset class that gets instansiated twice (once for the test, once for train) so they are functionally identical. I will try running now with the dataloaders swapped and see if it makes a difference. There is high demand on the cluster for the moment so I will get back to you when it completes.
Thanks.