@ptrblck Thank you for the reply! So if I understand it correctly:

if we just want to run a certain piece of code, we only need the CUDA runtime and match the right version between runtime, card arch and driver (and sometimes pytorch given its coverage)
whereas, we would need CUDA toolkit only when it comes to compiling some code, and the version of CUDA toolkit determines the runtime versions it can produce

Please correct me if anything is wrong. Thanks again!