mkserge:

But isn’t it odd? The *-runtime package claims to have cudnn installed (and we see it through torch.backends) but it’s not actually there?


It’s installed in the PyTorch binaries and is most likely linked statically.



 mkserge:

I install onnxruntime-gpu through pip, and it fails during import when it tries to load libcudnn and cannot find it.


This would mean that pnnxruntime-gpu doesn’t ship with its own statically linked cudnn, but is trying to dynamically link it from the system installation.



 mkserge:

I myself cannot find cudnn anywhere in the system, so pytorch must be doing something else here, no?


Yes, statically linking it and probably removing it afterwards to lower the size. If you need the local libs, you would have to use the devel container (or reinstall it into the runtime container).