Thanks @ptrblck !
Yeah, I see that
>>> print(torch.backends.cudnn.version())
8003

But I can’t find the lubcudnn binary anywhere in the container! 
root@fc13d70325fe:/# find / -iname 'libcudnn*'
root@fc13d70325fe:/#

The other dependencies of ONNX runtime are there though
root@fc13d70325fe:/# find / -iname 'libcublas*'
/opt/conda/pkgs/cudatoolkit-11.0.221-h6bb024c_0/lib/libcublasLt.so.11.2.0.252
/opt/conda/pkgs/cudatoolkit-11.0.221-h6bb024c_0/lib/libcublasLt.so
/opt/conda/pkgs/cudatoolkit-11.0.221-h6bb024c_0/lib/libcublas.so.11.2.0.252
/opt/conda/pkgs/cudatoolkit-11.0.221-h6bb024c_0/lib/libcublas.so
/opt/conda/pkgs/cudatoolkit-11.0.221-h6bb024c_0/lib/libcublas.so.11
/opt/conda/pkgs/cudatoolkit-11.0.221-h6bb024c_0/lib/libcublasLt.so.11
/opt/conda/lib/libcublasLt.so.11.2.0.252
/opt/conda/lib/libcublasLt.so
/opt/conda/lib/libcublas.so.11.2.0.252
/opt/conda/lib/libcublas.so
/opt/conda/lib/libcublas.so.11
/opt/conda/lib/libcublasLt.so.11

So, where is the libcudnn binary that pytorch is using?  
Edit: So I dug into the source code a bit, and it looks like pytorch has a completely separate implementation of cuDNN inside it’s own codebase. Is this true?