sbelharbi:

one question: can we compile the extension without gpu available, but it will be available later?
in some clusters, we have access to the frontal where there is no gpus. it is upon request that we are allocated gpus.


Yes, you can cross-compile for different architectures and create a pip wheel, but given the issues you were seeing in the past the JIT compilation might be the easier path for now.



 sbelharbi:

i assume the expensive time they are talking about is the loading when the compilation happens.
once loaded, the runtime of calling is the same as the ahead-method.


The “expensive time” is used during the initial compilation and the next call will use the already built and cached lib, so will be much faster. E.g. using this example I get an initial startup time of:
Loading extension module lltm...

real	2m13.153s
user	2m14.363s
sys	0m16.594s

and further calls need:
Loading extension module lltm...

real	0m1.984s
user	0m2.198s
sys	0m6.165s




 sbelharbi:

i read several times that when installing pytorch using conda install pytorch==1.9.0 cudatoolkit=11.1 -c pytorch -c nvidia, that cuda toolkit will be shipped with the installation.


No, that’s not correct, as only the runtime is shipped, not the entire CUDA toolkit.



 sbelharbi:

because the nvcc is part of the cuda toolkit and not the driver, i assume that nvcc is shipped with the installation as well. is that right?


No, it’s not part of the conda cudatoolkit binaries.