Solved by krishnavishalv in post #4 
              
              
                If you want to use default CUDA then installing from source is the only way I know of, because cuda is provided by default with conda/ pip wheel binaries.