you can try recompiling pytorch with cuda kernels runtime compiled on demand, I think config line like TORCH_CUDA_ARCH_LIST=“6.0+PTX” would do that. First run will be very slow, then they’ll be loaded from disk cache. I’m not certain that this will have a big effect though.
I’ve also heard something about “static runtime” somewhere on github, not sure if anything materialized from that.