It is possible that something could have changed without me knowing about it, since it is not my server. However the nvidia-smi output is still what I posted above. Perhaps I should try reinstall pytorch? Is there another Cuda test I could run?
The only thing that I tried to do was use another conda environment with the latest version of pytorch, version 1.6.0 with cuda 10.1. This latest pytorch version had the same problem unfortunately.