y_data is the same as x_data in this case but shifted by one word.  Allows me to calculate the loss by predicting the next word and checking if it turned out to be that word.
But you’re right – fair point. This is actually a minimalistic example that I made for asking my question - and almost certainly that’s why I was getting bad results; I’ll add back in the other features to see if they help.  In my actual network, I’m feeding in other features that would allow the network to distinguish between the <GO> token of input_1 vs the <GO> token of input_2.