Just adding to @SimonW’s answer to show the gradient:
>>> import torch

>>> x = torch.randn(1, device='cuda:0', requires_grad=True)

>>> x
tensor([0.6692], device='cuda:0', requires_grad=True)

>>> y = x.to('cuda:1') + 25

>>> y
tensor([25.6692], device='cuda:1', grad_fn=<AddBackward0>)

>>> y.backward()

>>> x.grad
tensor([1.], device='cuda:0')

>>>