Can a thread be assigned to each GPU?
In Torch, we can assign a thread to each GPU as like below. <SCODE>net = nn.DataParallelTable(1, true, true):add(net, {1, 2}):threads(function() require 'cudnn' cudnn.benchmark = true  cudnn.fastest = true end)
<ECODE>
Seungyoung_Park
unlike LuaTorch, we dont dispatch DataParallel via python threads in PyTorch. No, we plan to (and do) dispatch multi-GPU differently at high performance, without the user needing to do anything. This is irrelevant to the user, we are working on improving the internals and multi-gpu perf. Depending on how many parameters you have in your model, this is possible.
smth
Thanks for your reply.
Seungyoung_Park