In older versions of PyTorch, in order to move everything to the GPU, one had to do the following.
# Define a lambda at the top
cuda = lambda x: x.cuda() if torch.cuda.is_available() else x

x = Variable(cuda(torch.randn(10))) # When creating variables
model = cuda(Model()) # When creating modules

With the release of PyTorch 0.4, this has been slightly simplified as:
# Define the default tensor type at the top
torch.set_default_tensor_type(torch.cuda.FloatTensor if torch.cuda.is_available() 
                                                     else torch.FloatTensor)

# When creating tensors
x = torch.randn(10) # No need for additional bells and whistles

However, while creating modules, there is no such simplification:
device = torch.device('cuda' if torch.cuda.is_avaliable() else 'cpu')
model = Model().to(device) # One has to still cast the module to the appropriate device

Is there any workaround possible?