Your local CUDA toolkit will not be used, if you installed the conda binaries or pip wheels.
Did you properly select the desired CUDA runtime here 1, which should ship in the binaries? If so, what NVIDIA driver are you using?