Tokenizer encode to have an option to overflow from left
# 🚀 Feature request

Current tokenizer encode variants ( encode, batch_encode, batch_encode_plus) handle longer sequences than max_length by overflowing tokens from the right hand side and thus restricting the length to max_length. This feature request is to allow an option for the tokenizer encode methods to overflow tokens from the left hand side as well.


## Motivation

For problems dealing with dialog, if one were to train an intent classification or next sentence prediction model and the dialog was longer than max_length, one would like to throw away the tokens from the beginning of the conversation as they are less relevant than the more recent messages.

This motivates the need for a encoder that works well with dialog data where more recent tokens are more valuable.

## Your contribution

I could change the function `truncate_sequences` by adding a new truncation_strategy option that will truncate from left. But want to get feedback from the Huggingface team about this proposal.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>@misrasaurabh1 What solution do you use now for this encoding dialog data problem?<|||||>I use something like
`self.tokenizer.encode(input)[-self.block_size:]`
This throws a warning for length overflow so I deactivate it with logging. 
Also one has to make attention masks separately as some models require this.<|||||>Indeed, we should add an option to truncate on the left!
cc @n1t0 for our sprint of September.<|||||>perhaps add a truncation_side to https://huggingface.co/transformers/main_classes/tokenizer.html#transformers.PreTrainedTokenizer to be consistent with padding_side.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>@thomwolf @n1t0 Any plan for this? I just saw this because of the bot.

I think I can do this, seems like all the logic is here. 
https://github.com/huggingface/transformers/blob/07708793f20ec3a949ccab32cc4fe0c7272dcc4c/src/transformers/tokenization_utils_base.py#L2766

But how about fast 🤗 Tokenizers? Will I need to also change the rust code?

And I noticed something that might be a bug, and can be improved:

https://github.com/huggingface/transformers/blob/07708793f20ec3a949ccab32cc4fe0c7272dcc4c/src/transformers/tokenization_utils_base.py#L2816-L2831

Here it loops `num_tokens_to_remove` times to decide how many tokens needs to be truncated for each sequence, which can be calculated without looping.

And in case `stride` is not 0, it seems to return up to `stride`*`num_tokens_to_remove` extra tokens to `overflowing_tokens`.
https://github.com/huggingface/transformers/blob/07708793f20ec3a949ccab32cc4fe0c7272dcc4c/src/transformers/tokenization_utils_base.py#L2801-L2803
Also it seems weird to me that `overflowing_tokens` will be mixed with tokens from `ids` and `pair_ids`. Perhaps it should be a tuple of list if `TruncationStrategy` is `longest_first`.

Note to self: `overflowing_tokens` is used in squad to construct another pair if the doc is too long. `stride` is also used in squad. I can't find other use of `overflowing_tokens`.

https://github.com/huggingface/transformers/blob/969859d5f67c7106de4d1098c4891c9b03694bbe/src/transformers/data/processors/squad.py#L154-L216<|||||>One feedback about what's happening with this facility of left truncation being not available - its harder to use the datasets library and we have to do python Hackery which reduces the benefits of using the datasets library in the first place.<|||||>I recently needed to do exactly this, but ran into this issue so I had to manually truncate the text. Simply doing `encoded_tensor[-max_length:]` would also truncate samples that are less than `max_length` since they are padded to the right.  

Here's the approach I used instead:
```python
def encode_right_truncated(tokenizer, text, padding='max_length', max_length=512, add_special_tokens=True):
    tokenized = tokenizer.tokenize(text, padding=padding, max_length=max_length, add_special_tokens=add_special_tokens)
    
    if not add_special_tokens:
        truncated = tokenized[-max_length:]
    else:
        truncated = tokenized[0:1] + tokenized[-(max_length-1):]
    
    ids = tokenizer.convert_tokens_to_ids(truncated)
    
    return ids
```

Hope this helps future people finding this from Google/DDG<|||||>For anyone arriving here from search, note that this is now possible by setting [`truncation_side`](https://huggingface.co/docs/transformers/main/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.truncation_side) 
```python
# specify when initializing the tokenizer,
tokenizer = AutoTokenizer.from_pretrained(..., truncation_side = "left")
# or modify an already-initialized tokenizer, like
tokenizer.truncation_side = "right"
```
see https://github.com/huggingface/transformers/pull/12913