Hi, if you’re aiming to do distributed training across 2 GPUs you’d want to set world_size to 2 and the corresponding ranks would be 0 and. 1.
The GPU utilization is at 100% across both GPUs possibly because both GPUs are operating independently with no distributed coordination between them.
Your call to all_gather is the right approach and the error message confirms that world_size is set incorrectly. Fixing the world_size issue should unblock your use case.