One thing I do see is model = LSTMModel.LSTMpred(1,40).to(device) does not move the self.hidden over to GPU, because it’s not registered as a parameter or a buffer.
I think you want to move self.initHidden() into the forward function, rather being in the constructor.
Something like:
def forward(self, *input):
    hidden = self.initHidden()

The reason self.hidden doesn’t get moved to GPU is because it’s not an instance of nn.Parameter, or it’s not been declared as a buffer via https://pytorch.org/docs/stable/nn.html?highlight=register_buffer#torch.nn.Module.register_buffer 13 so PyTorch doesn’t know when you do .to('cuda') to move this onto GPU.