Solved by albanD in post #6 
              
              
                From nvidia-smi screenshot, you can see that there are 4 GPUs, however, the torch.cuda.device_count()=3, 

If you still have the CUDA_VISIBLE_DEVICES that hides the 4th GPU that is expected. Keep in mind that nvidia-smi does NOT respect this env variable and will always show you all the devices. 

T…