Based on the FAQ 39 it seems you might need to move the complete code to the if-clause protection.
Could you move the torch.cuda.is_available() calls into the if guard (i.e. the DEVICE assignment and setting cudnn.benchmark)?
If that doesn’t help, could you post the error message?