ncclInvalidUsage: This usually reflects invalid usage of NCCL library (such as too many async ops,
too many collectives at once, mixing streams in a group, etc).

This probably indicates something where we end up using NCCL incorrectly. One guess I have is that probably the environment variables for rank and world_size are not set up correctly when we use torch.distributed.run.