Not sure what’s going on, as the model seems to be on the GPU.
I would assume @Probe would get an error in forward, as it seems that
h_in = nn.Parameter(torch.randn(self.num_layer, batch_size, self.hidden_dim))
c_in = nn.Parameter(torch.randn(self.num_layer, batch_size, self.hidden_dim)) 

are still on the CPU, while self.lstm should be on the GPU.
Could you check that?