Thanks for your quick response.
Is it possible for PyTorch to use CUDA Driver API instead of CUDA Runtime API, thus PyTorch should have more control over how to load the kernel code.
This issue becomes a bottleneck when we try to deploy our product with multiple process, for 14 process, it takes 85%+ memory of 32G. It is ok for train, but when used it in production scenarios, it is an issue.