You could use a WeigthedRandomSampler, in case you could calculate and assign the weight for each sample corresponding to the folders. This would have the advantage to compute the weights only once and then sample from the two folders.