Why not the mlm use the information of adjacent sentences?

I prepare two sentences for mlm predict the mask part:"Tom cant run fast. He [mask] his back a few years ago." The result of model (uncased base) is 'got'. That is meaningless. Obviously ,"hurt" is better.
I wander how to make mlm to use the information of adjacent sentences.
The model is already using adjacent sentences to make its predictions, it just happens to be wrong in your case.

If you would like to make it choose from a specific list of words, you could use the code that I mentionned in #80. <|||||>Thanks @rodgzilla !<|||||>What 's the mlm accuracy of pretrained model? I find  the scores of candidate in top 10 are very close，but most are not suitable. Is this the same prediction as Google's original project?