I think the problem comes from the different data types of X (float64) and mu (float32). When you do X = torch.FloatTensor(X), you change the type of X to float32, hence the correct result. If you specify the type of mu to float64 and do not change X, everything should also work. As @ptrblck said, Variable is deprecated so here is an example of what I have with PyTorch 1.0
    N = 10
    D = 5
    K = 2
    X = np.random.rand(N, D)
    X = torch.from_numpy(X)
    mu = torch.randn(D, K, dtype=torch.float64)
    res = X[0, :] - mu[:, 0]
    print(X[0, :])
    print(mu[:, 0])
    print(res)