It would be great to test your code without using to method.
I still do not know which line causes this problem. Actually it should not bother you to convert your tensors to GPU because you are getting your items from Dataset class and they are by default on CPU. Or you have some codes somewhere which overrides these rules.