Here is an example of my suggestion:
torch.manual_seed(0)
targets = torch.randint(0, 5, (10,))
u, c = torch.unique(targets, return_counts=True)
print(u, c)
> tensor([0, 2, 3, 4]) tensor([1, 2, 4, 3])
classes_to_keep = u[c >= 3]
print(classes_to_keep)
> tensor([3, 4])

idx = []
for class_to_keep in classes_to_keep:
    idx.extend((targets == class_to_keep).nonzero())
idx = torch.tensor(idx)
print(idx)
> tensor([2, 4, 7, 9, 0, 1, 5])

print(targets[idx])
> tensor([3, 3, 3, 3, 4, 4, 4])

You could then use the idx in a Subset.