The hidden state at the start of each batch is not always zeros.
If you want to preserve some signal between batches because you are modeling one large sequence in chunks like a language model you would keep the hidden state at the end of batch_1 feed it as the initial state to batch_2 and so on. You have to detach it in between batches so that you do not compute gradients wrt to operations that happened in batch_1 while working on batch_2. In keras you can do this with the stateful=True flag.
If you don’t need any signal between batches because you are modeling many separate sequences like you would in document classification. You do not have to worry about manually feeding the hidden state back at all, at least if you aren’t using nn.RNNCell. you should use the lstm like this:
x, _ = self.lstm(x)

where the lstm will automatically initialize the first hidden state to zero and you don’t use the output hidden state at all.
again, the hidden state is not the output you should be using for classification.
x = (self.hidden[0] + self.hidden[1])/2

^-- is a large problem if you have not removed it already.
Also the loss functions in pytorch are meant to work with either logits or log_softmax. I would really recommend spending some time with the world language model again. The example is quite good.