BERT does mask-answering or sequence prediction or both???
I'm working on a exciting project but need to know something fast.

I seen BERT adds a word at the end of your input, like sequence prediction, elongating your input text.

But I read that BERT has been trained at (and is a pro at) filling in the blank word mask, and in fact CANNOT do sequence prediction at the end of one's input (contradiction!!). When BERT adds a fill-in answer where the mask sits, it goes in context.

I want a tool that returns me a similar word in context (the mask ANSWER!!), but the colab tool I tried is not a mask/answer giver, ..it adds words at the endddd of my input lol!!!
I'm not sure I understand your question.

But one thing BERT can do is mask-answering indeed (guessing a word in the middle of a sentence).

BERT is quite bad at doing sequence prediction at the end of an input because it's not trained on partial sentences.<|||||>When BERT fills-in a MASK, does it always use a answer it has seen or does it generalize and create never-seen Q-As? Ex. "The *dog* in the hat, by Dr. Souse the Cat"<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
