Well that depends on how large each op is. If they are big enough, we already paralellize each op.
In the case where they are super super small, yes you would gain from a parallelization on top.
But to get back to the main question:

we don’t have a map
I don’t think there is any plan to make one

It would be a lot of work
It would have limited benefit has it would only help very small, cpu-only tasks.


If your ops are small, you don’t want to use CUDA