Thanks for the input.
I’ll follow up with internal teams on what the next debug steps are.
It seems that the communication is somehow failing, but I’m not sure what might be causing this.
Could you post more information about your system? I.e. is it a custom workstation or a specific pre-configured server?
EDIT:
Are you seeing the same issue using different devices?
E.g. cuda:0 to cuda:2 etc.?
Also, for the failing case, could you add torch.cuda.synchronize() before printing b?