I have 2gpus in my system. I am using Dataparallel module over my model and I have made my both of the gpu visible using os.environ[“CUDA_VISIBLE_DEVICES”] = “0,1”. But, while running only 0 is selected if zero is the first in visible devises entry or 1 if  it is the first in entry.  What could be reasons why both of gpus wasn’t used together?
Then I tried manually creating replicas. But, I am getting error
RuntimeError: torch/csrc/autograd/variable.cpp:115: get_grad_fn: Assertion output_nr == 0 failed.
This happens at my LSTM part of my code
replicas = t.nn.parallel.replicate( model , device_ids)
inputs = t.nn.parallel.scatter(( dropout, encoder_word_input, encoder_character_input,
decoder_word_input, decoder_character_input,
None), device_ids)
replicas = replicas[:len(inputs)]
outputs = t.nn.parallel.parallel_apply(replicas, inputs)
out = t.nn.parallel.gather(outputs, output_device)