I was hoping to be able to do something like:
def make_tensor():
    return torch.tensor([1.,2.])

if not ARGS.disable_cuda and torch.cuda.is_available():
    device = torch.device(f"cuda:{ARGS.gpu}")
else:
    device = torch.device('cpu')

with torch.device(device):
    t = make_tensor() 

and have this return a tensor on either the cpu or the specified gpu. However, this might also have the problem of putting lots of unnecessary things on the gpu. I didn’t know about the option to set the default type, and if you think that’s not a good idea, then maybe what I’m suggesting isn’t either.