How did you install PyTorch? Did you build it from source or installed the binaries?
Since the binaries ship with CUDA and cudnn, you would just need to install the NVIDIA driver.
Note that also nightly binaries are built, in case you need the latest version.