Question: Token sequence length longer maximum sequence length
## ❓ Questions & Help

I'm using `run_glue.py` with a task name of `SST-2` to fine-tune a binary classifier on my data, which I put into the required format. However, some of my data's sentences are longer than the `max_seq_length` of `512` for `BERT` and `RoBERTa`; so, I get

`WARNING - transformers.tokenization_utils -   Token indices sequence length is longer than the specified maximum sequence length for this model (length_of_my_string > 512). Running this sequence through the model will result in indexing errors`.

What exactly is happening here? Are the training examples with more than `510` tokens still being used? If so, is the string being truncated down to `[CLS]` + the `first 510 tokens` + `[SEP]`? Is there any way to increase the `max_seq_length` or implement something like `head+tail`, which selects the `first 128` and the `last 382` tokens like suggested in this [paper](https://arxiv.org/pdf/1905.05583.pdf). That paper also uses `discriminative learning rate` as suggested [here](https://arxiv.org/pdf/1801.06146.pdf). Is there any plan to implement this?


Going through the source code, the sequence is actually truncated.

https://github.com/huggingface/transformers/blob/079bfb32fba4f2b39d344ca7af88d79a3ff27c7c/transformers/tokenization_utils.py#L846-L853

The warning occurs because `encode_plus` calls `convert_tokens_to_ids` _first_ and only then the IDs are truncated. The warning originates from `convert_tokens_to_ids` before truncation has happened. This is quite confusing indeed, since in the end result the IDs _are_ truncated.

Perhaps one of the maintainers can chip in.<|||||>> Going through the source code, the sequence is actually truncated.
> 
> https://github.com/huggingface/transformers/blob/079bfb32fba4f2b39d344ca7af88d79a3ff27c7c/transformers/tokenization_utils.py#L846-L853
> 
> The warning occurs because `encode_plus` calls `convert_tokens_to_ids` _first_ and only then the IDs are truncated. The warning originates from `convert_tokens_to_ids` before truncation has happened. This is quite confusing indeed, since in the end result the IDs _are_ truncated.
> 
> Perhaps one of the maintainers can chip in.

So, it safe to use or not?<|||||>This should have been patched in release 2.2.0.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
