I am very sorry I did not express my question clearly.
In fact, we have four data sets. In a forward and backward step, the model needs two batches which from two of the four data sets. Moreover, the model goes through four data sets in an epoch. The batches data is not repeated in an epoch.
The pseudo code is as follows:
For step in range(100)
Randomly sample two data sets. (like dset A and dset B);
Load two batches (like batch a and batch b) from these two data sets; # In next round, if we sample dsetA and dsetB, the batch a’ and batch b’ is not the same as batch a and batch b. Finally, we need go through all samples in four datasets. 
Feed into the model;
End