It is not about caching, what I tried to explain is that to call a kernel you either transfer a pointer to tensor data (address of 8 bytes) or a scalar (1-8 bytes), it is the same overhead, probably unavoidable (unless you would compile specialized kernels with less function arguments).