The pip wheels and conda binaries ship with their own CUDA runtime (as specified during the installation), so you would only need a corresponding NVIDIA driver to run your code. Your local CUDA toolkit will be used if you are building PyTorch from source or a custom CUDA extension.
The docker image would ship with a PyTorch source build which used the specified CUDA toolkit. The -devel container also comes with the CUDA compiler etc.


“can I install pre-built pytorch labeled by any CUDA version” - it depends on the GPU you are using (e.g. Amperes needs CUDA>=11.0) as well as the driver (it has to be sufficiently new).


See 2.


Check the CUDA compatibility docs 6 for a detailed overview.