I have about 10 GPU host indexes to be run on distributed mode. I need to use all the GPU machines available. But the problem is that torch.cuda.device_count() returns 1. I verified that the environment variable do have proper values ( 1,2,3,4,5,6,7,8,9,10 → indicating all 10 device indexes) .
Can anyone tell me whats going wrong here? Really appreciate your time.