I have multiple datasets, say 2 datasets.
And they have different training scheme and different labels.
So, I think I can’t concatenate 2 datasets into single dataset.
So, I’m thinking to create 2 custom dataset classes.
Then, what is the best (or correct) way to train a single model?
# Let's say epoch is 10
for i in range(10):
  # For example, I first train network over one whole dataset
  # (like using 10000 pairs of images)
  for imgs,lbls enumberate(one_data_loader):
    train_network(imgs,lbls)
  # Then, I should train network over another whole dataset?
  # (like also using 10000 pairs of images)
  for imgs,lbls enumberate(second_data_loader):
    train_network(imgs,lbls)

Is this correct training scenario?
Because I guess if I train network over one large dataset,
it means parameters are tuned based on that dataset,
and if I train tuned network over different large dataset,
I guess current network is tuned based on current dataset,
with disappearing effect of training over first dataset.
I wonder whether my guess is correct.
If so, I wonder what’s the correct training way for this case.