The PyTorch binaries do not ship with nvcc, so you won’t be able to compile other libraries with it.
The solution would be to install the same CUDA version locally on your machine as is used by PyTorch or build PyTorch and the other lib from source using your system-wise CUDA install.