Unfortunately, no updates as I cannot reproduce this issue and haven’t seen it before.
I would recommend to:

update PyTorch, the NVIDIA driver, etc. to the latest version
try different environments, such as the official PyTorch docker container or the NVIDIA NGC one, which uses a newer NCCL version
run the PyTorch tests 1 (in particular the CUDA tests) on both GPUs and check, if you are seeing some numerical issues

If that doesn’t help, I wouldn’t exclude a hardware defect.