You could try to change the default tensor type to a device tensor via e.g. torch.set_default_dtype, but I would generally not recommend to try to push all tensors to the device, but use the explicit approach instead.
The error should show the line of code which is failing and which tensors are included, so check the .device argument of these tensors and make sure they are on the GPU.