Yet, I am not sure how using gloo  instead of nccl  may affect performance

NCCL is much more performant than GLOO when dealing with GPU tensors (we’ve seen a difference of 1.5x - 1.8x in some cases). If performance is important for you, you should try to stick with NCCL and get it to work.