In my case, I am trying to train one model with two different dataset.
But the model failed to recognize the test data from the second dataset at all.
My flow is as following:

model = initializeFunction()
optimizer = optimizer(model.params)
train for some epochs using the dataset1 with optimizer
test with dataset1
optimizer_1 = optimizer(model.params)
train for some epochs using the dataset1 with optimizer_1
test with dataset2