Try this torch.cuda.set_device(0); This will put everything to GPU by default.

Expected object of backend CPU but got backend CUDA for argument #2 ‘target’


The error says that the loss was expecting something to be on CPU but rather it was found to be on GPU.
So share your loss function maybe and the way you are calling it (might help others to help  )