helloworld2021:

but the operation torch.DoubleTensor(self.weight_mat(x)) can only be done on cpu.


This operation is explicitly creating a CPUTensor, so it’s not a limitation of where this operation can be executed, but what it’s used for.



 helloworld2021:

However, I don’t fully understand your solution.


What the user tried to do seemed to be a transformation of the float32 tensor to a float64 tensor (you could ask him to double check).
This can be easily done via the tensor.double() operation, which neither will change the device (no device mismatch) nor will it detach the tensor.
Your approach creates a new parameter (which won’t be optimized, as it’s depending on the input x, is recreated in each iteration, and is thus unknown to the optimizer), which will also detach the operation from the computation graph and won’t change the dtype to float64, which seems to be the original use case.