Are you able to create tensors on each device?
Could you run this code and check the device ids?
x = torch.randn(10)
for id in range(4):
    y = x.to(id)
    print(y.device)

Also, note that the device ids start with 0, in case you would like to use the “first two” GPUs.