If you install the binaries, your local CUDA installation won’t be used and the shipped cudatoolkit will be used instead.
You would only have to make sure your driver is up to date or has at least the minimal version as seen in this table 1.
You can check the driver version via nvidia-smi.
Your local CUDA 9.0 version (or rather the nvcc compiler) would be used, if you are building PyTorch from source or a custom extension.