I am training different models on different GPUs.
I have 4 GPUs indexed as 0,1,2,3
I try this way:
model = torch.nn.DataParallel(model, device_ids=[0,1]).cuda()

But actual process use GPU index 2,3 instead.
and if I use:
model = torch.nn.DataParallel(model, device_ids=[1]).cuda()

I will get the error:

RuntimeError: Assertion `THCTensor_(checkGPU)(state, 4, r_, t, m1, m2)’ failed.  at /data/users/soumith/miniconda2/conda-bld/pytorch-cuda80-0.1.8_1486039719409/work/torch/lib/THC/generic/THCTensorMathBlas.cu:230

How to specify the GPU usage with index?