My guess is that on Gloo backend, all the data from different GPUs need to somehow first implicitly gather the data to GPU 0 and then transfer it to CPU.
Please not that as also summarized in Distributed communication package - torch.distributed — PyTorch 1.8.0 documentation 3
This is the rule of thumb:

Use the NCCL backend for distributed GPU training.
Use the Gloo backend for distributed CPU training.

Therefore, we should always favor NCCL on GPUs.