Does PyTorch install cuda?
CuDNN is another nvidia library and I’d say you should install it your self. It’s typically copied in cuda folder but if you want a system with several pairs cuda/cudnn you may save it in a different one.
Anyway im not an expert so consider this as a weak insight