colesbury:

OMP_NUM_THREADS=8


Thank you! After some experimenting, I get almost the same speeds with the MKL. OMP_NUM_THREADS=1 or 2 works best for me. The default setting slows it a lot. I wonder why would the OMP_NUM_THREADS be too high by default? My setup could be specific to my research, but it seems that it could be slowing other jobs too. I guess with cuda and cudnn, MKL takes on very small chunks of data and if you split it in many threads, the thread setup overhead is killing the job.