The DataLoader should provide the same random ordering when seeded and if there are no race conditions.
This example gives the same ordering for num_workers=0 or =1.
torch.manual_seed(2809)
torch.backends.cudnn.deterministic = True # Not necessary in this example

class MyDataset(Dataset):
    def __init__(self):
        self.data = torch.randn(25, 1)
        
    def __getitem__(self, index):
        print('Index: ', index)
        return self.data[index]

    def __len__(self):
        return len(self.data)


dataset = MyDataset()
loader = DataLoader(dataset,
                    batch_size=5,
                    shuffle=True,
                    num_workers=0,
                    pin_memory=True)

for batch_idx, data in enumerate(loader):
    data = data.to('cuda')

If you use a higher number of workers the order or the samples might differ.
At least I observe this effect on my machine. I assume this is due to the race conditions in different processes.