I use both.
For my topic i am interest in the hidden state of LSTM.
LSTM such as i implemented, thanks to pytorch, the cell state and the first hidden state are initialized automatically.
As you said, i get better results for every batch.