PyTorch has intra-operand and inter-operand parallelism. This means for a given op, you’d want not necessarily want to use all threads. If you have an application where you know you don’t need the latter, you can adjust the defaults.
Best regards
Thomas