I realize the op posted a long time ago but to keep people from getting confused by this.
These lines need to be removed



 AdityaAS:

x = (self.hidden[0] + self.hidden[1])/2
x = x.squeeze(0)


also these.
edit:
also remove this
self.hidden = self.initHidden()
from the forward pass, it’s reseting your hidden state to zero every time you make a forward pass.
and really just refer to the example language model for the rest! https://github.com/pytorch/examples/blob/master/word_language_model/model.py 177