I think you need
return F.softmax(out), hidden

otherwise you won’t be able to forward the state the following step.
And, if it’s likely you’re going to train it with a Cross Entropy Criterion, then output just the logit-state tuple (out, hidden).