Hi, I have loaded my 2 submodules in 2 GPUs - one per GPU as below
submodule1.cuda(0)
submodule2.cuda(1)
and in the forward pass
x1’ = submodule1(x1)
x2’ = submodule2(x2)
In the above case, GPU1 is idle while GPU0 is processing. Is it possible to run the above 2 in parallel and not one after the other?