The latest PyTorch release is currently 1.7.1 and I would recommend to stick to it instead of falling back to old versions.
Your local CUDA installation will not be used to run PyTorch, as the conda binaries and pip wheels ship with their own CUDA runtime. You would have to make sure to use the appropriate NVIDIA driver on your system to use the CUDA runtime.
Your local CUDA toolkit would be used, in case you want to build PyTorch from source or any custom CUDA extensions.