Hello Thomas, thanks for your help!
I tried what you proposed and converted the model and the input to doubles, I also removed the softmax operation to help narrow down the problem. So now the model output on PyTorch 1.0.1.post2 is:
tensor([[ 3.0757,  5.4147, 15.5268,  ..., -3.8933,  1.6677, -0.2514]],
       dtype=torch.float64, grad_fn=<AddmmBackward>)

And on PyTorch 0.4.1:
tensor([[ 2.7830,  5.2725, 13.6867,  ..., -2.6902,  0.1903, -0.8445]],
       dtype=torch.float64, grad_fn=<ThAddmmBackward>)

Also I noticed that the model’s output when running on PyTorch 1.0.1.post2 seems to be non-deterministic:
Run 1->
 tensor([[ 3.0757,  5.4147, 15.5268,  ..., -3.8933,  1.6677, -0.2514]],
       dtype=torch.float64, grad_fn=<AddmmBackward>)

Run 2->
tensor([[ 3.2392,  7.3212, 15.5781,  ..., -3.5685,  0.5535,  0.7085]],
       dtype=torch.float64, grad_fn=<AddmmBackward>)

Run 3->
tensor([[ 5.2961,  6.7340, 22.7242,  ..., -4.9779,  0.6964, -1.0704]],
       dtype=torch.float64, grad_fn=<AddmmBackward>)


As you can see the differences are not at all small. What on earth is happening?
Could it be that F.dropout()  is not turned off by model.eval() on Pytorch 1.0??
Thanks in advance!