Pro tip: You don’t have to intialize the hidden state to 0s in LSTMs. PyTorch does that automatically.
That network looks fine imo. Maybe try changing the embedding size, stacked layers, and input_size.
You’re passing the hidden layer from the last rnn output. Instead you can using the output value from the last time step. Or maybe average out all the hidden/output values.
What’s your train accuracy?