I think you do not need to install CUDA separately because it is included in the binary pytorch distribution, but:

you need to have to proper NVIDIA driver installed for your GPU (I think it will probably have to be the latest NVIDIA proprietary driver, not the the default open-source neveau driver)
your GPU must be supported by the CUDA version which is included in the python binary installation. If the GPU is an older model, it may be that an older CUDA version still supports it but a newer version not anymore

I think the pytorch pages could make all this a bit clearer, when I first looked at the installation instructions I thought the cuda version to specify is the one pytorch expects to be installed, when in reality it is the one included in the binary distribution.