Thanks for your information. Actually I have tried
model.cuda()
model = torch.nn.DataParallel(model, device_ids=[0, 1])
but there is always the following errors:
File “/home/speech/treelstm_nlg/HRED/ContextLSTMLayer.py”, line 21, in forward
output, (hn, cn) = self.rnn(x, (h_0, c_0))
File “/home/speech/treelstm_nlg/venv/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 727, in _call_impl
result = self.forward(*input, **kwargs)
File “/home/speech/treelstm_nlg/venv/lib/python3.6/site-packages/torch/nn/modules/rnn.py”, line 579, in forward
self.check_forward_args(input, hx, batch_sizes)
File “/home/speech/treelstm_nlg/venv/lib/python3.6/site-packages/torch/nn/modules/rnn.py”, line 534, in check_forward_args
‘Expected hidden[0] size {}, got {}’)
File “/home/speech/treelstm_nlg/venv/lib/python3.6/site-packages/torch/nn/modules/rnn.py”, line 196, in check_hidden_size
raise RuntimeError(msg.format(expected_hidden_size, list(hx.size())))
RuntimeError: Expected hidden[0] size (1, 64, 300), got [1, 128, 300]
why for that?
By the way, what is the difference between DataParallel and DistributedDataParallel, I seems the official documents suggest more DistributedDataParallel. In my case, some computation code is written by myself, so I am not sure, which is the best choice.
Thanks in advance!! Really appreciate for any feedback.