Hi all,
I’ve made some progress. First of all, I want to explain the new things I’ve learnt about the number of threads.
There are two different possible multiprocesses in PyTorch, “intra_op” and “inter_op”. The difference between them are the elements involved in the operations. If I’ve understood it correctly, “intra_op” refers to the operations between elements in the same minibatch and “inter_op” refers to the operations between elements in different minibatches. You can read about that here. So, I’ve separated the election of number of threads in two “intra_op” and “inter_op”. Blelow you can see the new bechmark graph: