Thank you @engelnico. I think I am still confused on the reason we want to retain hidden state for different batch. Mainly because I read this https://discuss.pytorch.org/t/when-to-call-init-hidden-for-rnn/11518/7: 87

I think you should call
hidden = net.init_hidden(batch_size)

for every batch because, the hidden state after a batch pass contains information
about the whole previous batch. At test time you’d only have a new hidden state for every sentence so you probably want to train for that.

The argument in this comment is since we will use initial hidden states (say 0) in test time, we would want the model to learn that by implicitly resetting the hidden states for each samples (so the hidden states only pass across time, but not samples). Am I missing anything here? Thanks.