How large is the data tensor? If it is not large enough, the GIL contention across threads and the extra overhead of setting this up could overshadow the speed up brought by using multiple GPUs. Another thing is how did you measure the delay? As the computation is done on CUDA, you might need to use CUDA events and elapsed_time 1 to get the accurate measure.
If elapsed_time still shows no improvement, can you try:

increase chunk size.
use multiple processes