Thanks Simon, for getting involved!
I tried exactly what you said, but the libmkl_gf_lp64 was not found.
So I tried specifying the path to the one that Pytorch errored on, and the response was Unknown:
nm -an  /opt/intel/mkl/lib/intel64/libmkl_gf_lp64.so | grep mkl_lapack_ao_ssyrdb
U mkl_lapack_ao_ssyrdb

I tried shortening the value being looked for so I could see what is listed:
Chris_Palmer@FASTAI:~$ nm -an  /opt/intel/mkl/lib/intel64/libmkl_gf_lp64.so | grep mkl_lapack_ao_
                 U mkl_lapack_ao_cgeqrf
                 U mkl_lapack_ao_cgetrf
                 U mkl_lapack_ao_cgetrfnpi
                 U mkl_lapack_ao_cpotrf
                 U mkl_lapack_ao_cspffrt2
                 U mkl_lapack_ao_cspffrtx
                 U mkl_lapack_ao_dgeqrf
                 U mkl_lapack_ao_dgetrf
                 U mkl_lapack_ao_dgetrfnpi
                 U mkl_lapack_ao_dpotrf
                 U mkl_lapack_ao_dspffrt2
                 U mkl_lapack_ao_dspffrtx
                 U mkl_lapack_ao_dsyrdb
                 U mkl_lapack_ao_inquiry
                 U mkl_lapack_ao_sgeqrf
                 U mkl_lapack_ao_sgetrf
                 U mkl_lapack_ao_sgetrfnpi
                 U mkl_lapack_ao_spotrf
                 U mkl_lapack_ao_sspffrt2
                 U mkl_lapack_ao_sspffrtx
                 U mkl_lapack_ao_ssyrdb
                 U mkl_lapack_ao_zgeqrf
                 U mkl_lapack_ao_zgetrf
                 U mkl_lapack_ao_zgetrfnpi
                 U mkl_lapack_ao_zpotrf
                 U mkl_lapack_ao_zspffrt2
                 U mkl_lapack_ao_zspffrtx

I looked for all instances of libmkl_gf_lp64.so, and found many candidates.
When I execute nn -an against any of these I get no response coming back… is that bad?
On the most likely candidate I tried again the first part of the name to get near:
nm -an  /home/Chris_Palmer/.conda/envs/fastai/lib/libmkl_gf_lp64.so | grep mkl_lapack_ao_
U mkl_lapack_ao_inquiry

If any of this seem to be OK, what’s the optimal way to tell the system to use the correct one?
/home/Chris_Palmer/.conda/envs/fastai/lib/libmkl_gf_lp64.so
/data/home/Chris_Palmer/.conda/envs/fastai/lib/libmkl_gf_lp64.so
/anaconda/lib/libmkl_gf_lp64.so
/anaconda/pkgs/mkl-2018.0.0-hb491cac_4/lib/libmkl_gf_lp64.so
/anaconda/pkgs/mkl-2018.0.2-intel_1/lib/libmkl_gf_lp64.so
/anaconda/pkgs/mkl-2018.0.2-1/lib/libmkl_gf_lp64.so