I had done something similar before (can’t remember why though).
I had made my own loader inheriting from dataloader who returned an iterator that I also made myself, inheriting from the normally returned iterator.
At that iterator’s next() method, I caught the stopiteration exception and then reshuffled indices for the next iteration over the entire dataset.
Plus that, add a shuffling in the initialization and you’re good to go.
Something like this in Python

class _DataLoaderIterWrapper(_DataLoaderIter):
    def __next__(self):
        try:
            super().__next__()
        except StopIteration:
            # DO STUFF
            raise StopIteration


class DataLoaderWrapper(DataLoader):
    def __iter__(self):
        return _DataLoaderIterWrapper(self)