thanks for the explanation.
so  out2 = F.conv2d(in2, weight2) will start before F.conv2d(in1, weight1) is finished on GPU 1, right?
BTW, what if the weights are shared in two lines, should I do it by:
out1 = F.conv2d(in1, weight.cuda(0))
out2 = F.conv2d(in2, weight.cuda(1))