Yes, it seems like the error is in 1.6.0 (tried in the server). Tried 1.5.0, that gives expected results too.
1.6.0
>>> import torch
>>> torch.__version__
'1.6.0'
>>> fc = torch.nn.Linear(2, 2)
>>> t = torch.tensor([[0.5, 0.3]])
>>> print(fc(t))
tensor([[-0.1817,  0.4468]], grad_fn=<AddmmBackward>)
>>> print(fc(t.cuda()))
tensor([[0.1775, 0.5823]], grad_fn=<AddmmBackward>)
>>>

1.5.0
>>> import torch
>>> torch.__version__
'1.5.0'
>>> fc = torch.nn.Linear(2, 2)
>>> t = torch.tensor([[0.5, 0.3]])
>>> print(fc(t))
tensor([[0.0865, 0.0900]], grad_fn=<AddmmBackward>)
>>> print(fc(t.cuda()))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/anaconda3/envs/pytorch1.5/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/ubuntu/anaconda3/envs/pytorch1.5/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 87, in forward
    return F.linear(input, self.weight, self.bias)
  File "/home/ubuntu/anaconda3/envs/pytorch1.5/lib/python3.8/site-packages/torch/nn/functional.py", line 1610, in linear
    ret = torch.addmm(bias, input, weight.t())
RuntimeError: Expected object of device type cuda but got device type cpu for argument #1 'self' in call to _th_addmm
>>>