Hello,
I run my model on a single GPU, the result of the test set is normal. But when I change to run on multiple GPU, the metrics evaluated on the test set (loss, RMSE,…) are not stable, they increase dramatically over time. I wonder why this problem happens, I thought the result must be the same in both cases.
Pytorch version: 0.4.1.
Best,
Khang Truong.