CUDA and cudnn was installed by an admin.
I checked the debian installation, and now this CuDNN 7.4.1 (built against CUDA 10.0) message makes total sense to me:
dpkg -l | grep cudn
ii  libcudnn7                                                  7.4.1.5-1+cuda10.0                         amd64        cuDNN runtime libraries
ii  libcudnn7-dev                                              7.4.1.5-1+cuda10.0                         amd64        cuDNN development libraries and headers

I hadn’t checked that before because I assumed there can’t be a mismatch when PyTorch 1.1.0 binaries run on CUDA without any issues.
I installed an appropriate cudnn version and now it works, thanks a lot!
Out of curiousity: Do you know why the previous PyTorch 1.1.0 binary installation was working? Does the binary contain cudnn?