Good point with detach()!
What is actually the benefit and intuition of carrying the hidden state over? Say I have do just sentence classifications, and each sentences are independent. Why should one batch depend on the final hidden of a previous batch. Here, re-initialization seems to me the more consistent method.
Sure for a language model where you break long documents into chunks, so the chunks depend on each other, carrying the hidden state over seems the more natural way to go.