Ok many thanks once again @iffiX
With parallelizing over the 2 GPUs, I am thinking I guess I can use this:
device = torch.device("cuda:0")
X1 = X1.to(device)

device = torch.device("cuda:1")
X2 = X2.to(device)