Disclaimer: The following are just my thoughts. I’m neither in expert in deep learning nor PyTorch. It’s just one area related to my research work.

I only compute the loss (and back propagate) for the last word of the sentence

I’m rather sure that you backpropagate through the whole sequence. The for loop in your train() method should build the backgraph, so when you call backward() it should consider the whole graph.

I’ve seen math explanations, […]

Not sure what you mean by this last paragraph. You use the RNN for classification, so there’s no notion of “take the word 1 and predict word 2, then take words 1 and 2 and predict word number 3 and so on”. Since sounds more like sentence generation with some kind of decoder architecture.