@sbsky Either technique comes with its own overhead. If the time it takes to launch one of these kernels is >> the time it takes to execute it, you’ll have to optimize the launch itself. If you decide to do this with multiprocessing, you’ll need to move the references to those tensors between processes, which isn’t free. The alternative is to launch fewer kernels, which is what @mcarilli did in AMP.