Actually I used a variety of Intel CPUs: 3 or 4 Intel i7 Gen 6,7, and 8. I used Cuda 8.0, 9.0, 9.1, cudnn 7.0, 7.1 and Nvidia GTX 1080ti GPUs. In all cases the GPUs are all severely underutilized when MKL is used. If I switch to openblas, the performance improves.
I know that MKL is supposed to be faster than openblas, but in all instances I get much slower performance with pytorch - CPU usage spikes 3-5x and GPU utilization drops 2-3x. This is counter intuitive and I spent a lot of time trying to figure it out. Could it be a bug?