blade:

So what do you think is wrong with the parallel implementation that takes double as much time to run with two GPUs?


This is surprising to me. I would assume at least the CUDA ops can run in parallel. Could you please share the full code used in this test? We will investigate.