It will
>>> x = torch.randn(1, device='cuda:0', requires_grad=True)
>>> x
tensor([-0.1979], device='cuda:0', requires_grad=True)
>>> x.to('cuda:1').backward()
>>> x
tensor([-0.1979], device='cuda:0', requires_grad=True)