Does the following code uses multiple cores?
import torch

a = torch.rand(100, 1000, 1000)
b = torch.rand(100, 1000, 1000)

while True:
    c = torch.bmm(a, b)

If so, try upgrading to 0.4 via pip to check that it is not a build problem from the old version.