How to specify GPU usage?
I am training different models on different GPUs. I have 4 GPUs indexed as 0,1,2,3 I try this way: <SCODE>model = torch.nn.DataParallel(model, device_ids=[0,1]).cuda()
<ECODE> But actual process use GPU index 2,3 instead. and if I use: <SCODE>model = torch.nn.DataParallel(model, device_ids=[1]).cuda()
<ECODE> I will get the error: How to specify the GPU usage with index?
zhoubinxyz
I am using Ubuntu 16.04. The GPU indexing are the same as you have. <SCODE>CUDA_VISIBLE_DEVICES=2,3 python xxx.py
<ECODE> In addition, I don’t think that dataparallel accepts only one gpu.
Seungyoung_Park
zhoubinxyz
apaszke
I am using pytorch 0.1.9 and Ubuntu 16.04. When I use CUDA_VISIBLE_DEVICES=2,3 (0,1), ‘nvidia-smi’ tells me that gpus 0,1 (2,3) are used. I do not know the reason, but the gpu id used in nvidia-smi and the gpu id used in pytorch are reversed. You can check it if you use Ubuntu 16.04.
Seungyoung_Park
shicai
albanD
zhoubinxyz
I’m curious about this as well. Can you currently use fractional GPU usage as in tensorflow? The tf equivalent is something like this: <SCODE>    with tf.device(FLAGS.device):
        gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=FLAGS.device_percentage)
        sess_cfg = tf.ConfigProto(allow_soft_placement=FLAGS.allow_soft_placement,
                                  gpu_options=gpu_options)
<ECODE>
jramapuram
<SCODE>dtype = torch.FloatTensor
# dtype = torch.cuda.FloatTensor # Uncomment this to run on GPU
<ECODE> Also, do I really have to track how GPUs are assigned, I am fine with torch just doing its stuff automagically. In particular I would love to see how: is made into a GPU version of it.
Brando_Miranda
Dear All, I am getting the below error: RuntimeError: cuda runtime error (2) : out of memory at /opt/conda/conda-bld/pytorch_1518244421288/work/torch/lib/THC/generic/THCStorage.cu:58 I think pytorch is not communicating with the Nvidia GPU, please advise.
saurabhjha21
ptrblck
Yes you are correct, it was fine after i restart the machine
saurabhjha21
for a Unix command soln you can also do: <SCODE>export CUDA_VISIBLE_DEVICES=$i
<ECODE> though of course that only works if the scripts are independent and stuff like that…otherwise the other solutions here are probably better…
Brando_Miranda
CUDA_VISIBLE_DEVICES=$i python main.py
JannaShen
hi， do you have the answer？
labixiaoK
Is there anyone who knows that… When I attach below code in python file(in main.py), <SCODE>import os
os.environment["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environment["CUDA_VISIBLE_DEVICES"] = "0"
<ECODE> The former one doesn’t specify(divide) GPU but, the latter one works well. It seems strange to me. Thanks ahead.
DonghunP
I wouldn’t recommend the first approach, since you would have to make sure these lines of code are imported before any other library, which might take the GPU. If some script imports PyTorch and these lines are executed afterwards, they won’t have any effect anymore. The second approach makes sure to mask the devices before running the Python script.
ptrblck
Totally understand thanks!!
DonghunP
LucasX