Thanks @osalpekar.
OpenMP environment variables dont make a difference if torch.set_num_threads is already set.
In fact I use GOMP_CPU_AFFINITY to enforce a particular set of openMP threads to run on specific cores. I did play also with OMP/MKL_DYNAMIC set to false.
I still do not understand though what torch.set_num_threads controls if I end up having 1 thread for FWD and 1 different thread for BWD. And several other threads.
I may think that there is a total amount of work and some “empirical” definition of how many threads to have for certain amount of work. the env var will overwrite that heuristic.
Just trying to find the code where that is precribed/defined.
Are you aware of a document where it is described the architecture in terms of thread/functionality ?
Why would I get > 6 threads if I have torch.set_num_threads set to 1.
Regards.