Hi,
I set all seeds to the same value.
I have run the code several times on those two servers with both single gpu mode and two gpus mode. The results showed that the performances are both far away from each other in these two modes.
I wonder if it is caused by adam algorithm?