Solved by ptrblck in post #2 
              
              
                You are creating new tensors on the CPU in this line of code: 
weight_prod = torch.DoubleTensor(self.weight_mat(x))

If you want to change the data type, use out = out.double() instead and make sure it’s the right type for all further calculations.