Solved by vdw in post #2 
              
              
                Here’s what I think(!) is going on: 
You want to train a network that takes a sequence of words as input to predict the next word (many-to-one). But what you’re actually training is a many-to-many network, more specifically a sequence tagging network like for POS or NER tagging. 
Given your example,…