How about using a try … except block for each of the two loaders so that when each one of them runs out, it can get reset independent of the other one. So here I’m giving an example with tow loaders loader_A and loader_B:
# Loading a batch from A:
try:
     x_a, y_a = next(data_iter_A)
except:
     # loader A needs to reset
     data_iter_A = iter(loader_A)
     x_a, y_a = next(data_iter_A)

# Loading a batch from B:
try:
     x_b, y_b = next(data_iter_B)
except:
     # loader B needs to reset
     data_iter_A = iter(loader_B)
     x_b, y_b = next(data_iter_B)