Actually what I decided to do is to have 10 different dataloaders, each one correspond to each classes. Then I randomly select between the dataloaders. if you can share the critical part of  the sampler code that you have written may be I can try your way too.