If you have 4 cores and need to do, say, 8 matrix multiplications (with separate data) you could use 4 cores to do each matrix multiplication (intra-op-parallelism). Or you could use a single core for each op and run 4 of them in parallel (inter-op-parallelism).
In training, you also might want to have some cores for the dataloader, for inference, the JIT can parallelize things (I think).
The configuration is documented here, but without much explanation: https://pytorch.org/docs/stable/torch.html#parallelism 610
Best regards
Thomas