I tried to debug your code and I think this problem is related to this issue 170 (I just tested the CPU side so far).
You’ll get the same results, if your calculate the MSELoss manually:
torch.mean((output32 - X_float_var)**2)
> tensor(0.2477)
torch.mean((output64 - X_double_var)**2)
> tensor(0.2477, dtype=torch.float64)

loss(output32, X_float_var)
> tensor(0.1844)
loss(output64, X_double_var)
> tensor(0.2477, dtype=torch.float64)