@ptrblck @smth
I found it is time-consuming on calculating weights for this WeightedRandomSampler(weights, sample_size).
As people said, len(weights) == len(dataset). so if the dataset is huge (10^9), this part is inefficient
for index in range(len(dataset)):
    reciprocal_weights.append(class_prob[dataset[index][1]])
weights = (1 / torch.Tensor(reciprocal_weights))

Is it really necessary to calculate all weights (10^9) if my sample_size is just very small (e.g. 1000)?
I don’t think so, especially the weights array usually is just a vector with lots of redundancy which can be express as the sparse vector.