The problem is that the article is not very well document, so it’s not easy to see the important details without knowing what to look for. Let me try to make the comparison between your approach and the one in the article.
Let’s assume you have a long text based on which you want to build your language model, e.g.:
A B C D E F G H I H K L M N O …
Your approach is not wrong
out,_ = lstm(input)
out = out[:,-1,:]

but only assumes the training data of the form (input sequence, next word). So you training data might look like:
A, B
A B, C
A B C, D
B C D, E
C D E, F
…
(assuming a max. length of 3 for the input sequences)
The method in the article does something different. The important details is # The targets, shifted by one. So here the dataset looks like:
A B C, B C D
B C D, C D E
C D E, D E F
…
If you take all outputs of the LSTM – i.e., not just after the last step – you implicitly get (A, B) after the 1st step, (A B, C) after the 2nd step and (A B C, D) after the 3rd/last step for the first data item (A B C, B C D).
There’s no fundamental difference between both approaches, but the 2nd is more efficient.