I find it easier to create tempraries once and pass them to kernels (as scratch space), also I write independent kernels and use a flat index and grid-stride loops; but if you have to, you can use dynamic parallelism.
I also asked about efficiency of temporaries 31, and it seems to be Ok to just create them on the fly when needed. I tested it, and so far it looks like PyTorch’s custom cuda allocator does a great job when it comes to memory allocations.