Hi. For example, for the code snippet
a = torch.rand(3).requires_grad_()
l = a.sum()
torch.distributed.all_gather([a,b,c], a)

l.backward() will trigger an error say

one of the variables needed for gradient computation has been modified by an in-place operation

I assume this is because all_gather do in-place change for all a,b,c.
But why? As a is emitted by the current process, it’s not necessary to change it and cause this issue. Is there any consideration behind this all_gather behavior?
Thx