According to my understanding, calling init_hidden() once every training epoch should do the trick, however it (hidden weights) must be updated for every sentence, so that updated weights would be used and they wouldn’t be changed to zero for every sentence as “init_hidden()” initializes weights to zero.
Also it should be kept in mind that only the latest set of weights be retained and not all of them. We wouldn’t want to keep all of the training weights.
So, according to me,



 torcheror:

hidden = repackage_hidden(hidden)


This can be used after every batch, but



 torcheror:

hidden = net.init_hidden(batch_size)


This should only be used once an epoch,