Ok, got it.
It seems that your datasets are somehow loaded using __img_factory. You just need to pass the name to your dataset_manager and it will create the appropriate dataset for the passed class?
If so, can you create two separate datasets for your two classes?