I can reproduce it using the code sample.
Could you please create an issue 4 to track this bug?
As a workaround you could use a SubsetRandomSampler and pass the shuffled indices to it.
Inside the Dataset.__getitem__ you might need to create a single index tensor via:
index = torch.stack(index)
x = self.data[index]

since the SubsetRandomSampler will pass a list of tensors to the Dataset for the BatchSampler approach.