Warning/Assert when embedding sequences longer than positional embedding size
Hi team, love the work.

Just a feature suggestion: when running on GPU (presumably the CPU too), BERT will break when you try to run on sentences longer than 512 tokens (on bert-base).

This is because the position embedding matrix size is only 512 (or whatever else it is for the other bert models)

Could the tokenizer have an assert/warning on it that doesn't allow you tokenize a sentence longer than the number of positional embeddings, so that you get a better error message than a bit scary (uncatchable) cuda error.



Could do that indeed Patrick.
In particular when the tokenizer is loaded from one of Google pre-trained model.
If you have a working implementation feel free to do a PR.
Otherwise I will have a look at that when I start working on the next release.<|||||>Happy to do a PR :) will do today or tomorrow