Thank you again for your answer. Yes, the idea is to train two different models (A, B) in two GPUs (1, 2) at the same time but load the best model obtained during training. Hence, I want to load the best correct model of A naturally into a new instance of A in GPU1.  The same applied to B. I saved and loaded models as suggested here but I did not save and load a general checkpoint.
The models (A, B) were saved into different folders to avoid issues. After analysing a little more, I think that the problem is likely to be related to one configuration file which has all values of parameters the programs need (collected during runtime).