Ok your code doesn’t look wrong. I think the problem might be that you just need to run it for longer because as long as the loss is decreasing it should be fine. I think the accuracy on predicting sentence order would not change much until it has trained for a while. Sorry I couldn’t help more.