I am using pytorch 0.1.9 and Ubuntu 16.04.
When I use CUDA_VISIBLE_DEVICES=2,3 (0,1), ‘nvidia-smi’ tells me that gpus 0,1 (2,3) are used.
I do not know the reason, but the gpu id used in nvidia-smi and the gpu id used in pytorch are reversed.
You can check it if you use Ubuntu 16.04.