If you need to compile the package, you would still need a local CUDA installation and I doubt you will be able to install it in Colab through the notebook (I might be wrong).
Thus I think you would need to use a proper server and either install CUDA directly on it or via docker etc.