I would suggest the use of a Sampler. This is an optional argument when initializing a DataLoader.
In general, it’s expected to yield sample indices to the dataloader. A very straight forward example would be something like this:
class CustomSampler(Sampler):
    def __init__(self, data_size):
        self.samples = [i for i in range(data_size)]
    
    def __len__(self):
        return len(self.samples)
    
    def __iter__(self):
        for sample in self.samples:
            yield sample

Here it simply generates a list of all possible indices and yields them 1 at a time in order. What you could do is randomly shuffle the list initially and use the same sampler when you wish to preserve order. Here would be an example:
class CustomSampler(Sampler):
    def __init__(self, data_size):
        self.samples = [i for i in range(data_size)]
        random.shuffle(self.samples)
    
    def __len__(self):
        return len(self.samples)
    
    def __iter__(self):
        for sample in self.samples:
            yield sample

Notice that the shuffling is done upon initialization. So if you created two dataloaders using the same instance of CustomSampler, the order of samples will be the same. I would recommend reading the doc for more info