If both tensors are already on the GPU, the result will also have the save device:
a = torch.randn(10, device='cuda:0')
b = torch.randn(10, device='cuda:0')

c = torch.stack((a, b))
print(c.device)
> device(type='cuda', index=0)