Could you check the memory usage on the server via nvidia-smi and make sure you have enough device memory left?
If you are using multiple GPUs, you can specify a specific via
x = torch.randn(1, device='cuda:id'), where you would have to replace id with the device ID.