Solved by ptrblck in post #2 
              
              
                I think your issue might be related to this one. 
Try to pass the weights for each sample to WeightedRandomSampler and see if it’s working.