Often the most efficient way to use a GPU is to leverage a single kernel at a time doing a lot of work. If the utilization of your GPU is high with this approach, then there likely isn’t a way to speed things up (trying to parallelize things by running multiple kernels in parallel would likely slow things down with the increased contention). If the utilization is low, you might see if you can increase the amount of data parallelism (e.g., batch size).