Another option is to code the __len__() method of your Dataset 3 class to return 5000. Then in the class you can have a map from the index received by __getitem__() to an index in the 50,000 data. For example, using uniform random sampling.