+1 to everything Chris says above. A few other thoughts.

h1 = 32: you say only 3 classes but your output is 32. I’ve seen this in other models also. I don’t understand the use of these extra outputs.
MSELoss: I could be wrong about this so feel free to correct me. I thought MSELoss was more appropriate for true value prediction vs a classification task. I would use crossentropy as it includes a softmax. The LSTM module utilizes a tanh activation fuction so I “think” you can end up with negative activations which may cause problems with the summation within the MSELoss. As I said I could be wrong about this so anyone feel free to correct me. Something I wanted to research further.