That’s strange. I tested on another two GPUs now and everything is OK. Since the two GPUs which I meet the problem is training now with PyTorch 1.4. After the training is over, I’ll test whether the problem occurs by accident.