Hm, I’ve never seen a language model being set up like this then – obviously doesn’t mean it’s wrong, though!
Most of the time X is a sequence of current words and y being the next word, i.e., a many-to-one model. Here, the batch size never changes. Now I can kind of see that your code combines this to a many-to-many.