Have you initialized the hidden state of your lstm layer? Also you probably need to get the hidden state at each iteration of your training and use it in the next forward() call, otherwise you might be starting from a new hidden state at each iteration (and your model will not converge)
I would suggest to take a look at the word language model example at the pytorch github