tlim:

I am wondering whether is there some sort of general guides on CUDA, PyTorch, Driver, GPU Versioning to memory needed for context allocation. This could be a much easier thing to follow than trying out unique compilations.


You could try to see some theoretical requirements which would depend on the actual hardware setup, but given that each PyTorch, cuDNN, cublas, CUDA, NCCL version might add/remove kernels for a specific compute capability, the easiest way it to profile the CUDA context directly.