I’m not sure that ConcatDataset is intended to be used with two datasets that have completely different classes. You might want to iterate through it without training the model as a sanity check to see you do get 20 classes. After checking that it should be easier to take a look at the TestLoader and see what is causing the mismatch between the reported length and the actual length.