Should n’t this follow the policy or behaviour we have in Numpy or at least CPU Torch.The operations in Numpy and Torch CPU are also in parallel but we only have more parallelism in GPU compared to CPU. Torch in CPU at least with those experiments that I have preformed results  the same as Numpy.