Thanks for your answer, ptrblck! I thought of that, but I am not keen on the idea of loading the state_dict on the other machine, I would rather go with a more programming approach/solution. On the other hand, I want reproducible results and the model is kind of sensitive in its initialization, that’s why I asked for a “trick”.
Do you ever encounter something like that? My thought is that the behavior I report previously, is due to the different number of cores on each GPU, but I can’t get my head around it and think of a solution.
George