Solved by vdw in post #5 
              
              
                Well, I had a look at the code. While I’m not familiar with this setup for a Language Model (LM) either, a look at the training data made a bit clearer to me. Still, everything that follows are not much more than educated guesses. 
Most fundamentally, an LM aims to predicts the next word given an in…