The problem is that the GPU itself is not built to run independent tasks in parallel. So you won’t be able to get the same speed with many small ops as with a single big one.
If you run on CPU, you can use multithreading as you would do for any workload to parallelize your execution.