I see. It doesn’t make sense. If we create two random tensors with the same shape as the output of the model and the shape target, and pass them to the rmse_loss, it has no problem.
I think you can debug it line by line, use print statement to display the device of each tensor after each line of computation. Then, maybe we can understand where this problem is happening.