you can write a custom Sampler, and shuffle in a more fine-grained way. See the sampler keyword argument, instead of shuffle=True. You can see some of the samplers here: https://pytorch.org/docs/stable/data.html#torch.utils.data.Sampler 48
They are quite simple to implement, so you can implement your custom sampler that will be more aware of your dataset’s slicing that you want.