Since the sampler defines the sampling strategy, the shuffle argument would make the sampler meaningless.
Think about some sampler you’ve created to draw a sequence of images in a particular order. If the DataLoader could shuffle, the sequence might be broken. That is why the sampler now defines how to draw samples and when to shuffle.
In your example, you don’t need to specify shuffle=True, as SubsetRandomSampler automatically shuffles the data using the subset indices.