Solved by ptrblck in post #2 
              
              
                You can make certain GPUs visible via CUDA_VISIBLE_DEVICES=1,3,7 python -m ..., which would map GPU1, GPU3, and GPU7 to cuda:0, cuda:1, cuda:2 inside the script and execute the workload (DDP in your case) only on these devices.