Based on the description it seems that the test DataLoader is calling into the PRNG at one point. Since shuffle is set to False, the Dataset might still use it at one point. Could you post the Dataset implementation (for the testset in particular) as well as the test loop?