Thanks!
OK, after trying multiple setups, I managed to find a combination using pytorch=1.9 without conflicts. I will report tomorrow.
BTW, Is it fine that the cuDNN version does not match the one used for the Pytorch binaries?
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.5.0
[conda] pytorch                   1.6.0           py3.8_cuda10.1.243_cudnn7.6.3_0    pytorch

I did not have cudnn in the conda environment. I added that, and I will test a new environment without that discrepancy.