It depends on your purpose: if you want the hidden state to keep the information of the whole epoch, you can just initialize the hidden at the begin of the epoch; if you want every mini-batch from one epoch has the same initial hidden state, you should initialize the hidden at the begin of every mini-batch. However, the point why your code failed but the referential code succeed is the usage of ‘hidden = repackage_hidden(hidden)’ in the referential code: it will reconstruct a Variable for the next iteration. You can add one line
hidden = Variable(hidden.data)

below
output, hidden = rnn(data, hidden)

to make your code work.