Clearly, we need to manipulate the random seeding to achieve this, I have tried the one below, but I am getting different combinations in the two dataloaders of the same data set.
def random_seeding(seed_value):    
    if seed_value>0:
        np.random.seed(seed_value)
        torch.manual_seed(seed_value)   
        random.seed(seed_value)    
        if cuda: torch.cuda.manual_seed_all(seed_value)

Oh, and I am using next(iter...)  to get the batches from the two dataloaders, as follows:
img_batch1 = next(iter(dataloader_1))
img_batch2 = next(iter(dataloader_2))

img_batch1 is different from img_batch2!
NB. Each dataloader is using num_workers=1
NB2. Uasing state = torch.get_rng_state(),  before the first loader, then, torch.set_rng_state(state) before the second loader,  did not help neither