hmmm, how about you set batch_size=1 in the DataLoader, but your custom dataset itself returns a full batch everytime __getitem__ is called? That way you can carefully choose shuffling and other aspects by yourself in the Dataset.