Ah, OK…got it. Yes, in this case you use the LSTM as decoder.
Anyway, generating the words step by step is the way for any X-to-sequence model. You can, of course, during training time give the whole sequence to the LSTM in case of Teacher Forcing. But training the whole network using only Teacher Forcing gives you, I think, poor results. Teacher Forcing is only applied with a certain probability (e.g., 50%) since it has been shown to make the training more stable and faster.
During inference time you have generate your output sequence anyway step by step in a loop. I don’t that the loop is the bottleneck. Firstly, the heavy lifting is still done by the LSTM, and given a whole sequence to the LSTM just wraps the loop, but it’s still there.
All RNN-based decoders I have seen so far have the loop in their forward() method to process the words.