In the “normal” use case, you would have to push the model as well as your data and target tensors to the device manually and I would recommend to stick to this approach.
It’s clearer in my opinion than e.g. setting the default tensor type to a CUDATensor.