The conda binaries and pip wheels are not yet built with cudatoolkit=11.2 and you would have to use 9.2, 10.1, 10.2, or 11.0 as given in the install instructions 95.
Since these binaries ship with their own CUDA runtime, you would only need a local NVIDIA driver corresponding to the CUDA runtime you are selecting.
In case you want to build PyTorch from source or any custom CUDA extensions, you should install a matching local CUDA toolkit.