Edit: Never mind - MKL_DYNAMIC and OMP_DYNAMIC would not help so removed it from my post
I was wondering about PyTorch internals: if I am using a server with larger number of cores (64), would you recommend setting OMP_NESTED flag to true or false?
https://software.intel.com/en-us/mkl-linux-developer-guide-mkl-dynamic 22